An example of Go based e2e benchmark for various Kubelet operations without spinning up whole K8s cluster.
Sometimes you want to test or benchmark simple operation on one of the components from bigger system. In my case I needed to test and benchmark
kubelet metric endpoint optimizations we made.
Usually to run newest
kubelet from source involves running full Kubernetes cluster, ideally on some external cloud provider virtual machines. In order to do so I would probably hit thousands of unrelated issues and bottlenecks only to test very simple operation on
kubelet which in calling
Instead of spinning full cluster I decided to use my dev machine, build
kubelet from source, put in container and use our amazing e2e framework (shameless plug) that supports among other things, programmatic, interactive test/benchmarking with built-in Prometheus monitoring integration. The challenge here is run
kubelet without any other component (e.g
kube-api) other than
It was not trivial for me, since I am not strictly a Kubernetes developer, so I had to reverse engineer many things. It might be not simpler for others too, so I decided to show how to do it in this repo. Enjoy!
- Linux machine
- Cloned https://github.com/kubernetes/kubernetes somewhere locally.
First of all run
bash build-kubelet.sh <local path to Kubernetes repo>. This should build kubelet from the source you have and put that in local
Check ./kubelet_bench_test.go which should be self-explanatory. It contains single Go test that you can run as usual using
go -v test ./.... This tests starts interactive docker environment with
Prometheus(and some docker CRI-O shim) and spams it with couple of
After that Prometheus UI should show up in your browser with relevant view on
You can kill test by finding test output line
Waiting for user HTTP request on http://<some local address> ...and going to this address.