As Kubernetes is gaining momentum, the community grows and with it, the public knowledge and code that can save us time by reusing public experience. Helm came into the picture and allowed us to use public packages that will spin off many commonly used services that we may require to run on our cluster. These abstractions are excellent, and you should avoid reinventing the wheel, but three are some concerns with using publicity available code.
The first concern is if the deployment is even valid. As I mentioned before, the whole field is still growing, and it is an evolving process. There is no guarantee that this Redis (and I will be abusing them quite extensively on this newsletter) Cluster implementation you found is production ready. Often, it was designed to support a particular use case that may not suit your individual needs.
And then there are security concerns, which are the focus of this post. When implementing a publicly available solution, or even if you just cherrypick some of the code, there should be a system around it.
The security-minded Code Review
Just like any other piece of code you add to your project, a code review process should take place. During the process, you will have, even to yourself, to defend the choices, offer alternatives and preferably, have a fresh set of eyes that can ask more questions.
A code review process also helps you by keeping the end at mind even when you are still shopping for a solution. You will ask the right questions, and it may even help reduce the effects of confirmation bias. I compiled a short list of questions you should be asking yourself about any new public Kubernetes related code that you want to bring in to the project:
- Does this code come from a reputable source? There are great Helm Charts available from the official Helm repository. Try looking at them first, but don’t trust them blindly.
- Is the project well maintained? When was the last commit? How many open issues there are? How many pending pull requests?
- Think about how they architected the solution, did they chose correctly? Should they use a Statefulset or a Deployment?
- Is this a dated way of doing things? Kubernetes keeps getting updated, and sometimes blog posts and code will have many workarounds to things that are now solved. An excellent example to this would be using Role Based Vault authentication instead of Kubernetes with Hashicorp Vault.
- When working with clusters, make sure the there is a proper authentication and traffic encryption method.
- Does the code support RBAC?
- Are the Docker Images that they are using safe and maintained? You can do the same code review for the docker images themselves.
- Is code being pulled from the outside and ran during the init process?
- How is logging implemented?
- Can it run without root privileges?
- Which ports are exposed an why?
This list is by no means complete, and I will keep it updated, but you get an idea of how a code review process looks.
I think going through an example may give you a better sense of things, so as I promised, let’s look at Redis. Now Redis has a few ways of running a cluster. I’m going to focus on the Redis Cluster as opposed to the Sentinel implementation for simplicity, and because I am biased to show you a bad example. If you head over to the official Helm repo, you can find a working Redis Cluster Chart: [https://github.com/helm/charts/tree/master/stable/redis]
- It seems like it came from a reputable source.
- The project is well maintained.
- The architecture seems right We can move on to the actual files: The best place to start is the Values.yml file.
- I can see that they have an option for running with a password. So we do have a form of authentication.
- It doesn’t seem like there is a certificate option. Worth taking a closer look.
- Yes, the template files also don’t show any option for that.
- That means that there is no encryption between the master and slaves, nor the clients. (part of the point I was trying to make and will dedicate a full post on Redis) Next, we should look at the template files:
- They do seem to have RBAC support
- There is a network policy in place.
- How about the templates/metrics-deployment.yaml file?
- Wait, they are passing the password as an environment variable? That’s not good.
As you can see, we already found a few red flags in this project even though it is somewhat official. Again, it’s not the best example because we expected to find security issues with Redis, but it does illustrate the process.
Security is just one aspect to look at when reviewing Helm Charts and Kubernetes code, so make sure to read the Should I use this Helm Chart article.