1# Benchmarking logging
2
3Any major changes to the logging code, whether it is in Kubernetes or in klog,
4must be benchmarked before and after the change.
5
6## Running the benchmark
7
8```
9go test -v -bench=. -benchmem -benchtime=10s .
10```
11
12## Real log data
13
14The files under `data` define test cases for specific aspects of formatting. To
15test with a log file that represents output under some kind of real load, copy
16the log file into `data/<file name>.log` and run benchmarking as described
17above. `-bench=BenchmarkLogging/<file name without .log suffix>` can be used
18to benchmark just the new file.
19
20When using `data/v<some number>/<file name>.log`, formatting will be done at
21that log level. Symlinks can be created to simulating writing of the same log
22data at different levels.
23
24No such real data is included in the Kubernetes repo because of their size.
25They can be found in the "artifacts" of this
26https://testgrid.kubernetes.io/sig-instrumentation-tests#kind-json-logging-master
27Prow job:
28- `artifacts/logs/kind-control-plane/containers`
29- `artifacts/logs/kind-*/kubelet.log`
30
31With sufficient credentials, `gsutil` can be used to download everything for a job directly
32into a directory that then will be used by the benchmarks automatically:
33
34```
35kubernetes$ test/integration/logs/benchmark/get-logs.sh
36++ dirname test/integration/logs/benchmark/get-logs.sh
37+ cd test/integration/logs/benchmark
38++ latest_job
39++ gsutil cat gs://kubernetes-jenkins/logs/ci-kubernetes-kind-e2e-json-logging/latest-build.txt
40+ job=1618864842834186240
41+ rm -rf ci-kubernetes-kind-e2e-json-logging
42+ mkdir ci-kubernetes-kind-e2e-json-logging
43...
44```
45
46This sets up the `data` directory so that additional test cases are available
47(`BenchmarkEncoding/v3/kind-worker-kubelet/`,
48`BenchmarkEncoding/kube-scheduler/`, etc.).
49
50
51To clean up, use
52```
53git clean -fx test/integration/logs/benchmark
54```
55
56## Analyzing log data
57
58While loading a file, some statistics about it are collected. Those are shown
59when running with:
60
61```
62go test -v -bench=BenchmarkEncoding/none -run=none .
63```
View as plain text