Conf42 Cloud Native 2023 - Online

Kubernetes Secret Management The Right Way With Vault

Video size:

Abstract

Kubernetes Secrets by default aren’t secure. In fact, some Kubernetes documentation itself even recommends taking a look at a third-party utility. In short it’s because secrets are stored in plain-text by default. In this session, you’ll learn how to fix that issue by combining Kubernetes and Vault

Summary

  • All right, so let's go ahead and break into this PowerPoint a little bit. Really quick theory, just because like we all know youll need the theory to understand the hands on piece. So let's just go through it, but I promise it won't be too much PowerPoint hell.
  • Kubernetes does have the ability to create secrets. But they are unencrypted by default. They are stored in plain base 64 text. You should probably go with a third party if possible.
  • hashcore vault can be implemented in Kubernetes. Here's how to get it installed in your environment. And now we're ready to start injecting secrets.

Transcript

This transcript was autogenerated. To make changes, submit a PR.
All right, so let's go ahead and break into this PowerPoint a little bit. And it's going to be super straightforward. Really quick theory, just because like we all know youll need the theory to understand the hands on piece and you need the hands on piece to understand the theory, both them go together. So let's just go ahead and go through it, but I promise it won't be too much PowerPoint hell. All right, so first things first. What are secrets? Now youll usually hear that secrets are like passwords or API keys or something along those lines. But in reality secrets are really anything that you don't want in plain text. Now this could be a username for whatever reason, don't know a specific use case, but maybe you would want a username and a password to be tied together and not in plain text. Again, for whatever reason you could go ahead and you can put that into a secrets now out of the box. Kubernetes does have the ability to create secrets. In the core API group there is a resource that you can use to create secrets. However, here's the problem with Kubernetes secrets. Now, they are unencrypted by default. And what I mean by that is when you create a Kubernetes secret, it's stored in ETCD, which is the Kubernetes data store. And then from in there it's unencrypted. So there is zero encryption inside of ETCD for the secrets. They are stored in plain base 64 text. Which means if you have access to etCD, if a service account has access to etCD, if any engineer has access to etCD, if any attacker gets access to etcD because it's not secure for whatever reason, yes, they can get direct access to your secrets. So because of that, when I'm in production environments, I typically say that we should be going with another secrets provider, another third party. In fact, there is documentation even in Kubernetes documentation that states all of this. It's like, listen, everything is unencrypted by default. You should probably go with a third party if possible. Now from a third party perspective, there are a lot of different options. However, I typically go with Hashcorp vault. And the reason why, other than the fact that you can kind of just deploy it and not pay anything for it, there is so much documentation and so many getting started guides and so many different ways to run it in production that you can read about and watch videos on and even courses and all that good stuff. So when I'm consulting for customers, I'm always thinking about not what the latest and greatest is, but what's actually supported heavily. And Kubernetes and Vault is supported heavily. And I think the fact that it's supported, number one speaks to how good it is. But number two, it goes into just how important, having documentation and having best practices and all of that is. So with that, let's go ahead and learn about how we can implement hashcore vault in our Kubernetes environment. All right, so now that we know why we would want to use vault, let's go ahead and actually figure out how to get it installed in Kubernetes. So first things first, we're going to go ahead and we're going to use helm. Helm is the Kubernetes package manager. And essentially instead of just going to the raw Kubernetes manifest and running Kubectl apply or Kubectl create, we can version it with helm, update it, roll it back, et cetera. So first things first, you want to make sure you have the Hashicorp repo. Now I already have that because I have it installed already. So next what I can do is I can use this install command. Now I'm going to be naming my chart vault. I'm going to be calling upon the vault helm chart and then I'm going to use two flags. Well actually a few. So the first is to enable the UI. The second is to enable the service type as load balancer. And then next what I'm going to do is I'm going to specify the vault namespace and I'm going to create that namespace if it's not already created. All right? And as we can see, this was deployed. So I'm going to clear that screen and I'm going to run Kubectl getall namespace vault. Okay, so notice here how there are a few things going on. I have the services, I have the deployments and then the pods right here. So this container is still creating. Now we want to wait until it's in the running state so I can just show this to you really quick. All right, it's still creating. Let's go ahead and give it a few more minutes or a few moments should only be seconds. All right, so as we can see, it is running, but it's not officially available yet. It's saying zero of one. So what we want to do is I'm going to open up, well rather I'm going to go to this IP address and if you're wondering, by the way, I'm running on aks. You could be running in any cloud, any managed Kubernetes service, doesn't matter. I just so happen to be running in aks. So really quick, let's just go ahead and show that again. I'm going to copy the IP address which is the load balancer for the vault Ui. And then I'm going to head over to the web browser really quick. All right. And I'm going to pop that in and then I'm going to put it over 8200. Okay? So we can see the UI is up, but it's not unsealed. That's why you see that zero one there that the container is running, but it's not available. So what I want to do here is I am going to run this Kubectl exec command. And essentially what I'm doing is I'm running the vault operator init command inside of the container or inside of the pod rather without actually executing into the pod. So as you can see, we have a few unsealed keys right here. All right? And then here's the initial root token. So what I'm going to do is I'm going to have to run this command three times because we're going to have to unseal it a few times. So let's go ahead and run that just like this. And then I'm going to pop in the first key. And as you can see, it says one of three. So with vaults you always have to put in at least three unseal keys. So we have to do the same thing even though it's running in Kubernetes. Same thing. All right, so now let's put in the second one. All right, now it says two of three. And then finally let's put in the third here. Got to run the command first. All right, and as you can see, we are now good to go. So if I go back to the UI here, we can now see that it's asking me to sign in. So what I'm going to do is I'm going to scroll up again, I'm going to get that initial root token, I'm going to pop it in here. And we are now in and good to go. However, one thing that I do want to show is remember this pod used to just show running, but zero out of one was actually ready. So now if we run Kubectl, get pods namespace vault, we can see that it is now officially running. So our UI is up, Vault is deployed in our Kubernetes cluster. And. And now we're ready to officially start injecting secrets. And that's how you can get vault up and running in any Kubernetes. Cluster, thank you so much for watching.
...

Michael Levan

Kubernetes and Containers Trainer, Consultant, and Content Creator

Michael Levan's LinkedIn account Michael Levan's twitter account



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways