Conf42 Observability 2023 - Online

Observability: the Key to Unlocking GitOps

Video size:

Abstract

Git is the source of truth for the system’s intended state, observability the source of truth for actual state. Internal observability allows the GitOps controller to identify configuration drift; external observability allows operations staff and other systems to identify changes in the cluster.

Summary

  • Alejandro Mercado, currently living in Mexico City. Proud member of the DevOps Institute as ambassador for Mexico. GitLab Hero and Dev Network advisory board member. Talk is an holistic approach by reference to the whole. If you wish to connect, be my guest.
  • observability is a management strategy focused on keeping the more relevant and important and core issues at or near to the top of an operations process flow. Metrics, traces and logs are the three pillars of observability.
  • Gitop uses git repositories as a single source of true to deliver infrastructure as code. In the middle you have the reconciliation loop. Why GitHub is important and web companies are adopting Githubs. We have improved security and compliance, increased collaboration and productivity. Faster deployments and increase reliability.
  • You have to install Argo CD in the Kubernetes cluster. Of course you can sync manually. But for demo purposes I just wanted to sync it by clicking this button to synchronize. If something is going wrong, you are going to see it pretty, visually, it's going to put in red.
  • GitHubs is currently in the stage of its germination. A lot of companies are adopting the Githubs way to do their releases and to manage our Kubernetes clusters. It's unlikely that GitHub will become something bigger than DevOps, it's complement DevOps.

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Thank you very much again, honor to be here. I am Alejandro Mercado, currently living in Mexico City. Well, I am going to talk about a little bit about githubs and observability. Let me share my presentation, my deck. So the name of my talk is observability, the key to unlocking it ups. So this is a little bit about me. I am the dev now doing DevOps living in Mexico City. Also proud member of the DevOps Institute as ambassador for Mexico, GitLab Hero and Dev Network advisory board member. So there is my link. If you wish to connect, be my guest. So well, I would like to start with this slide to mention with this warning that this talk is an holistic approach by reference to the whole. So I don't just want to talk about some specific topic, technical issue, pretty deep. If not, I would like to talk about to see that the forest instead of the whole tree or just one tree. So I would like to start with observability. So observability is. Well, there is not a single definition about observability, but observability is a management strategy focused on keeping the more relevant and important and core issues at or near to the top of an operations process flow. The term is also used to describe software processes that facilitate the separation of critical information from routine information. So seems like a little academic definition. So maybe when we contrast observability versus monitoring, we have a better understanding of what observability is. But I would like to, or some companies like to explain observability with these three pillars. Metrics, traces and logs are the three pillars of observability. So you can think of an observability in these three components. It's like the secret sauce. When you have this data, you can be proactive and you can observe your systems well, it can also refer to the extraction and processing of critical information at the highest level, architectural operation systems. As I said, when we put in contracts with monitoring, when we talk about monitoring, maybe you have been there diving in a locks, ocean, so tons of lines. It takes a while to get real and important information about what is going on with the system. So that's why we have these new tools. Actually, in most cases when we talk about monitoring, it's because we are doing a post martin analysis. So this is the main difference between observability. Observability is to be proactive, is to have this information to avoid that the system crashes or to have some problems. And even then the user report the issue or the problem we already know about it because we have this observability running end to end. So monitoring tools passively gather information, most of which turns out to be insignificant or we don't get the real important information. So this can drown operations, personnel and even AI tools in data versus observability that actively gathers data to focus on what's relevant, such as the factors that drive operations, decisions and actions. So in this site is observability and why it is important because we are going to see proactively our systems. These new terms is not that new, but a lot of companies are adopting Gitops because it represents some advantages. So Gitops in a pretty brief definition, GitHub uses git repositories as a single source of true to deliver infrastructure as code. So single source of true is the key definition. So you can see this image, this is Githubs in a nutshell. We are going to have our desired state in a couple of manifest. Well, I have to say that these GitHub tools add a control plane to kubernetes. So we have this repo, this Git repo. It can be GitHub, GitLab, any repo, any repository that you have, it doesn't matter, but you have the manifest in this repo. This manifest is the source of true for declarative code. So it's going to do a reconciliation when you are committing and releasing your code. This is the CI CD pipeline. So you are running a series of stacks resulting in the update of the runtime environment to match the source. So if there is discrepancy or a difference between the desired state and the actual state of the system, it's going to do the reconciliation loop. So submitted code checks the CI process well, the CD process checks and applies requirements for things like security, infrastructure as code or any other boundaries set for the application framework. All changes to code are tracked, making updates easy while also providing version control, shoulder roadblock be needed. So well, this is GitHub's blue subservability. I would like to recommend to take a look of this image. So you have the desired state and the current state, and in the middle you have the reconciliation loop. I think that this is the key or the middle or stone of GitHub. So we observe, and if there is a difference between these two estates, we adjust. So that is why observability is key for Githubs. We can do GitHub's observability. So why GitHub is important and web companies are adopting Githubs because we have several advantages from other release managing strategies. We have improved security and compliance, increased collaboration and productivity, decreased manual oversight, faster deployments and increase reliability. And by the way, there is this site that I totally recommend to take a look. You can enter to open HTTPs opengirups dev. So you have this initiative that it sets like the principles of what is open. So open Gitops is a set of open source standards, best practices and community focused education to help organizations adopt a structure standardized approach to implementing GitHub. I highly recommend to take a look of this site. So it's not about the implementation or tooling, it's like a neutral vendor site. So the principle of GitHub, this principle I have to say are extracted from the site. The principle of GitHubs is that it's declarative, so you don't do imperative command lines. So it's based on manifest files, like I said in a previous slide. So is a clarity a system managed by GitHub must have if desired state expresses the clarity is know Yaml is nowadays everywhere is versioned and immotable. That means that the desired is stored in a way that enforces immotability versioning and retains a complete version history. So it's similar like when developers make commits and they have this versioning and can do rollbacks, it's pretty like the same. That's why it is GitHub, it's pulled automatically. So it's pulled because there are another paradigms that are pushed. In this case, GitHub is pulled automatically. So the subware agents automatically pull the desired state declarations from the source, so it pulls the desired state. Do the reconciliation loops with the actual states. And if there is a mismatch it's going to do the healing. So it's continually reconciled. So subway agents continually observe actual system state and attempt to apply the desired state. So that's the way that you can have availability system. And it's, it's very easy to maintain even if you have a lot of, I mean if you have 100 of pods and your kubernetes cluster is too big, this tool even have some nice graphical user interfaces that you can handle the state and see if something is wrong, you can apply the reconciliation. So the demo, there is not no show me code. This is what developer says. So there is a lot of tools that are applying the GitHub paradigm like fluxd wear harness, Jenkins codefresh that actually uses Argo CD wave. Well in particular I'm going to do the demo in Argo CD. So Argo CD is a declarative GitHub's continuous delivery tool for kubernetes. So remember it's a controller plane running on kubernetes. It's another control plane running kubernetes so we can handle the releases. So at this point I'm going to show the site. So there is a pretty cool training. If you go to codefresh IO you can find some training. So I will highly recommend to take this training because our folks of coldfresh already did the setup. So it's already there. It's pretty easy just to follow the instruction that as you can see in my browser we have all destructions just to follow. I will just show one of the lessons, that is just to show them. So it takes a little bit to prepare the environment. Okay, so we have the environment. So as I said, go to this address learning cutfresh IO. So it's pretty handy, pretty useful. You can learn about ArgoCD from GitHub fundamentals to some more advanced topics. So it's pretty cool because as I said, we just have to follow the instructions. The first task is to fart this repository that we have on GitHub.com. So we have already, I can show, I have my, this is the. So as you can see Argo CD is already installed. You can handle GitHub with the command line interface, writing commands, but you also have this nice Argo CD UI. It's like pretty nice, pretty useful interface that you can see what is going on with your Kubernetes clusters and suffering is ramp. You are going to see here in red, some pod is failing or if we lose communication you are going to visually see immediately. And of course you can do something to try to repair or to do a rollback or to do something. So we are going to do a new app. The application name is whatever you want demo, the project is default, the repository URL is the certification examples acting as without the git and the path is simple app. The cluster is the default that we have to know. You must need a Kubernetes cluster already installed that thanks to our fault from cutfresh we already have there and the name space we can set up the default. So you can leave these other arguments empty for now. So, so that's it. You can click on create. So at this moment we are going to have our first application running on kubernetes and based on GitHub so you can see the application is out of sync. If you see the yellow indicator is out of sync. So you can sync manually. But the idea is that you are not going to wake up every morning to sync all the Kubernetes cluster. So it can be set up automatically. But for demo purposes I just wanted to sync it by clicking this button to synchronize, you know, remember the desired state with the actual state. So if there is a difference, a discrepancy between the two estates, it's going to throw an error. So it's processing. Of course in the simple app you can take a look of the manifest, because maybe you are wondering about well, what is going on in the deployment is using this manifest. Of course the deployment is pretty simple. It's just an application, trivial go web, an application in go in the go programming language. So we just have one replica. Maybe if I change this state and do the synchronization again, it's going to throw an arrow because now I am telling that the desired state is two or five or twelve. So this is just a pretty simple deployment file, JaMl. So you have this container that is running a pretty simple application. And the services, the other JAML is just exposing the application in the port 80 80. So nothing too complex, but I think it's pretty illustrative. So it takes a little bit to get the synchronization. But as you can see in this learning environment, we also have the shell. So a lot of these activities or tasks, we can also do that in line. By line, I mean entering the commands. It totally depends on what you prefer. So when you have this nice interface, maybe you want to take advantage of that. I just want to see if the deployment is already done because I think it's a little bit slow. I don't know if it's my connection. I have my deployment, my simple deployment, but it's not ready. The pod is not available yet, so we have to wait to the. As you can see that the view interface is pretty nice. I think it's pretty handy. If you have some problem, you are going to have this health status you can see in this side menu bar. The first is the sync status. It's unknown synced when it's green and out of sync. Well in this case the status is processing the synchronization. But as you can see here in the health status, if something is going wrong, you are going to see it pretty, visually, it's going to put in red the frame. So maybe you can do the rollback and you can do the rollback here in the Argo CD user interface or in the command line, it's totally up to you. Of course I didn't show that, but you have to install the Argo CD in the Kubernetes cluster. And an easy way to start Argo CD is with the helm package manager. I am going to go back to my slides waiting to have this already synchronized. But well it's just to give an idea of what we are talking about. So going back to my presentation, well if we can get any conclusion of this, well there is a lot of operations, there is the evolution of operations and it's quite a bit a lot of operations. So this is kind of fun because we started with Ops following DevOps, Devsecops, devsec ML Ops, Triceratops, that this is the joke. But there is another Ops, no data ops, GitHubs of course, AI Ops, no ops and you name it ops. So it's mean. But for sure GitHub is important thing. So you can see that a lot of companies are adopting the Githubs way to do their releases and to manage our Kubernetes clusters. So definitely GitHubs is currently in the stage of its germination. I mean it's starting, it's going to grow for sure. So you can see the numbers. So totally a good investment of your time. If you are already doing DevOps, the recommendation is just to learn another tool. I mean it can be tough because you're already handling a lot of tools, but GitHubs is going to be part of. Sometimes people ask about if this is going to replace DevOps. It's not a replacement, it's a companion. Gitops is neither better nor worse than DevOps. It's not the correct approach. Gitops without DevOps does not have a future. So you need DevOps to do Gitops. The scope needs to be changed and directed at 14 investigation and how the approach can add DevOps is just a cultural thing. Also GitHub is part of that. So it's unlikely that GitHub will become something bigger than DevOps, it's complement DevOps, I would like to say. So if you take something of this brief talk, the four takeaways will be you cannot do Githubs without observability. You know, observability is important to have all the time, to observe all the time our systems and git is the single sort of true for the system intended state. While observability provides a single sort of true for the system actual state. The third one is internal observability allows the GitHub's controller to identify configuration drift. External observability allows operation, staff and other systems to identify changes in the cluster. So cloud native observability, as I said, is a new skill and task you must add to your DevOps for sure. So thank you very much. This is all for me. So thank you very much. And I am going to be around here. This is my twitter. If you want to contact me, be my guest. So thank you very much.
...

Alejandro Mercado

Sales & DevOps Engineer @ KMMX

Alejandro Mercado's LinkedIn account Alejandro Mercado's twitter account



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways