I found myself last week looking at a bit of code in K8s which I thought I could make better, so I set about trying to understand how to clone, change and test it.
Luckily K8s has some good docs, trust these over me as they’re a great guide. This blog is more of a brain dump of how I got on trying with Devcontainers and VSCode.This is my first try at this so I’ve likely got lots of things wrong.
As the K8s build and testing cycle can use up quite a bit of machine power I didn’t want be doing all this on my laptop and ideally I wanted to capture all the setup in a nice repeatable way.
I’ve been working with OPA recently and using KIND to test things out. This works really nicely but when I started using the same approach in CI I saw some errors.
Digging into things you can see that the nodes of the KIND cluster aren’t “READY” when the CLI finishes up so you need a bit of extra bash foo to make the process wait on the READY status.
This monster line of bash does the trick:
JSONPATH='{range .items[*]}{@.metadata.name}:{range @.status.conditions[*]}{@.type}={@.status};{end}{end}'; until kubectl get nodes -o jsonpath="$JSONPATH" 2>&1 | grep -q "Ready=True"; do sleep 5; echo "--------> waiting for cluster node to be available"; done
In this example I’m also deploying a K8s operator this needs be to up and running before I can run the integration tests , a similar bit of bash ensures that’s true too:
JSONPATH='{range .items[*]}{@.metadata.name}:{range @.status.conditions[*]}{@.type}={@.status};{end}{end}'; until kubectl -n opa -lapp=opa get pods -o jsonpath="$JSONPATH" 2>&1 | grep -q "Ready=True"; do sleep 5;echo "--------> waiting for operator to be available"; kubectl get pods -n opa; done
If you put those all together I can have a nice make file which:
until kubectl get nodes -o jsonpath="${WAIT_FOR_KIND_READY}" 2>&1 | grep -q "Ready=True"; do sleep 5; echo "——–> waiting for cluster node to be available"; done
kind-deploy:
./integration/deploy/deploy.sh
# Wait for OPA to be ready
until kubectl -n opa -lapp=opa get pods -o jsonpath="${WAIT_FOR_OPA_READY}"2>&1| grep -q "Ready=True";do sleep 5;echo"——–> waiting for operator to be available"; kubectl get pods -n opa;done