Conf42 Cloud Native 2024 - Online

Scaling Secrets in Kubernetes: A Secure Multi-Cluster Approach

Abstract

A secret is any piece of information that you want to keep confidential, such as API keys, passwords, certificates, and SSH keys. The integration of External Secrets Operator and Sveltos enables a powerful solution for managing secrets in a multitude of Kubernetes clusters.

Summary

  • Mardente: How to manage secrets when you are dealing with a fleet of Kubernetes clusters. Secrets can be anything like APIs, keys, password, certificates, SSH keys. Only authorized user can access this information. There are a lot of benefits of using a secrets management system.
  • external secrets operator is one of those tools. It synchronizes the content that is present in the secrets in the secret management system into your Kubernetes cluster. Sveltos can programmatically deploy add ons and application in a fleet of clusters.

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Everybody. My name is Mardente, I'm a principal engineer at Cisco Systems and my talk today is about scaling secrets in Kubernetes, a secure multi cluster approach, essentially how to manage secrets when you are dealing with a fleet of Kubernetes clusters, data security is critical as we and that includes keeping secrets confidential. So first of all, what is a secret? A secrets can be anything like APIs, keys, password, certificates, SSH keys, and if those falls into the wrong ends, it can have serious consequences. Secrets management system address this problem by storing your secrets in a secure encrypted format, which makes it more difficult for attackers to steal them. And also the secrets management system also provides a simple and secure way for authorized user to use this information. So there are a lot of benefits of using a secrets management system. As I mentioned before, security secrets is never stored like in plain text. Secret is always stored like using strong encryption. And only authorized user can access this information. Convenience, because you have a single place where you can go and create, update or delete your secrets. Doesn't matter wherever you are, as long as you have access to the secrets management system, you can do that. And the secrets management system provides you with detailed audit logs so you know who has access what and when. And this is important because it helps you comply with security regulations. And also if an incident happens now you know where to start from, you know what information has been accessed by whom, and so you can start from us tracking from there. What if your applications are running in a Kubernetes cluster? How do you manage your secret? In that case, we listed the benefits of using a secret management system, which means clearly we want to use a secrets managing system. But the secrets management system does not run in your Kubernetes cluster. So you need an application running within your Kubernetes cluster that can access the information present in the secrets management system and makes it available to the application running in your Kubernetes cluster. External secrets operator is one of those tools. It's an open source tool. It integrates with AWS, SQL manager, Azure Control, Azure, Google SQL Manager, IBM Cloud and many more. And what it does, it synchronize the content that is present in the secrets in the secrets management system into your Kubernetes cluster. So let's see how that is done. In this example, we have a cluster and the external secret operator is running within this cluster and we do have a Google Cloud secret manager and the external secrets operator has been provided with the confidential and information to access the Google secrets manager. The external secret operator detects that there is a secret in the Google Cloud secret manager and syncs it into the Kubernetes cluster. And it does it by creating a Kubernetes secrets into the Kubernetes cluster. Now the content of this Kubernetes secret matches the content of the secret present in the Google Cloud secret manager, which means another ten point like any application running in the Kubernetes cluster can access this information. But the external secret operator doesn't stop here. Let's say that we update the secret in the Google cloud secret manager because that's the only place where we want to create, update or delete our secret. Now the external secret operator detects that a change has been made. It knows that at this point the cooperative secrets is out of sync compared with the content that is present in the Google Cloud secret manager. And so it updates. And now your Kubernetes secrets thanks to the external secret operator is back in sync with whatever you have in the Google Cloud secret manager. Now this works perfectly if you have one cluster, but what if you have a fleet of Kubernetes clusters to manage? Well you may think that you want to install external secret operator, deploy external secret operator in each one of the clusters that you manage, but that is the wrong approach. It's not scalable and error prone. Think about the case where first of all you have to configure the external secret operator like in times. But think about the case where you want to change the credentials used by the external secret operator to use to access your secrets management system. Now there are place where you have to go and update this information. So if we have a fleet of cluster, this is where Sveltos can come and help. Sveltos is a Kubernetes add on controller. So if you have a fleet of Kubernetes clusters using Sveltos, you can programmatically deploy add ons and application in this fleet of clusters. How does Sveltos work it run in management cluster. So you have a Kubernetes cluster which you use a management cluster. Sveltos is running there cluster profile is the custom resource definition which is used to tell Sveltos which add ons and application to deploy where on which clusters and then you have a fleet of Kubernetes clusters. Those can be cluster API power clusters. Veltos has built in support for clusters API power cluster which means that if you create a cluster using cluster API, there is nothing else that you have to do as Sveltos automatically detects those clusters and can manage application and advance in those clusters. But then you can have any other type of cluster, GKE cluster, Sivo cluster, rancher, any type of clusters you can easily register to be managing by Sveltos. The only things that Sveltos require is that there is a network connectivity from the management cluster to the cluster that you want to be managed by Sveltos. Cluster profile as I mentioned, it's the CRD that is used to tell Sveltos what to do and it contains like few sections in the spec. The clusters selector field is a Kubernetes label selector which selects all a set of clusters. In this case it's selecting all the clusters which have label environment set to a theme which stands for functional application. Then it has an helm chart section. In these helm charts you can put one or more helm charts and what Sveltos will do, it will take each one of those and will deploy those in every cluster matching the cluster selector. In this case we only have Caverno. So Sveltos will deploy Caverno version 2.6.0 in every cluster matching the cluster selector environment. Functional application. Then you have a customization ref section and you have a list of customization configuration. Here each one of those entry points to a directory which contains customized files. What Sveltos will do will run the customized SDK on top of those. We'll take the output which is set of Kubernetes resources and we'll deploy those Kubernetes resources in every cluster that matches the cluster selector. And finally you can put the Kubernetes resources you want to be deployed in a secret. So in a config map, both secrets and config map have a data section and so in that data section you can put one or more Kubernetes resources. You can use both Yaml or Json. What Sveltos will do we'll take the content of any secrets and config map that is referencing and will deploy in every cluster which is matching the cluster selector. Now another strong point for using Sveltos is that anything that you want to be deployed can be expressed as a template. And Sveltos has the ability of instantiating these templates using resources which are present in the management cluster at the time of deployment. So in this example we're telling Sveltos take the content which is present in the config map into the full name space called info. Know that this is a template and instantiate it using the content which is present in this secrets in the management cluster in the default namespace called autoscaler. So let's see how we can integrate together external secret operator on Sveltos to efficiently manage secrets in a fleet of Kubernetes clusters. Well, it's very easy to do. So we'll start creating one management cluster and in this management cluster we deploy both Sveltos and the external secret operator. Then in this example we have two managed clusters, one GK cluster and one cluster API power cluster. We have a Google Cloud secrets manager and we configure external secret operator to synchronize whatever information, whatever secrets is present in the Google cloud secret manager. So what external secret operator will do? In this case, it detects that there is a secret called Sveltos secrets in the Google Cloud secrets manager. It takes this information and creates a secrets in the Kubernetes cluster. In the management cluster, Sveltos has been instructed to watch for this information, take this information and pass it to every managed cluster. So at this point you are secret. You added your secret in a single place, the Google Cloud secrets manager and you're using the external secret operator to sync that information into your management cluster. And then you're using Sveltos to propagate this information to every cluster that you manage. What happens if the secrets gets updated? We go and update the secret in the Google cloud secret manager well, external secrets operator, as we saw, detects that there was a change and updates by updating the Kubernetes secrets management cluster. And now Sveltos detects that the external secret operator has updated the secrets that they manage and it takes the content and updates the content like in every managed cluster. So it's very easy and programmatical way of efficiently managing secrets in a fleet of Kubernetes clusters. If you want more info, you can check this Sveltos repo, this GitHub.com project, expanders and or the external secrets operator Repo, which is again on GitHub external secrets external secrets thank you so much. That's all for me. Thanks a lot.
...

Gianluca Mardente

Principal Engineer @ Cisco Systems

Gianluca Mardente's LinkedIn account Gianluca Mardente's twitter account



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways