Hi, my name is Ismael Hommani. I am cloud cloud native developer GDE Wescale. In today's talk we are going to present you with clement a comparison between Argo and Flux, which are two GitHub's tools in the Kubernetes landscape. First of all, we want you to be aware that it is not a basic comparaison we want to make, but something that go beyond a simple benchmarking. It's about describing these difference of philosophy when it comes to consider Githubs in the flux mind or ArgoCD mind. So our comparison criteria will first be based on the model, that is to say the different entities and relationship with one another. It will allows us to deeply understand how flux and ArgoCD consider the GitHub's approach and will give you the right insight when it comes to make the choose between those two tools. Then we will present you the different user experience you can expect. Is it easy to install, how to do it, how to update it, how to debug it, and so on. Last but not least, benchmarking will give you a quick overview of what kind of expectation you can have in terms of cpu and memory consumption. We wanted to compare flux and Argosidoi because when it comes to Githubs in Kubernetes landscape, we have two clear winners, ArgocD and Flux. They have a wide recognition and community and adoption. But the question remains, what tools should I install? And usually we see comparaison in terms of performance and not philosophy. That's why we wanted to compare Flux and ArgoCD today. Before comparing those two tools, let's recap what Githubs is. Traditionally, we consider the CI CD approach with a monolithic pipeline where a file will imperatively describe how two deploy inside a target platform, which means that the runtime will need to provide the different tools we need to interact with these platform. First point, let's say for instance Kubectl. Second point, we have to provide these tools with credentials inside the pipeline, which means potential data leakage. And third point, if we want to deploy on another platform, we would have to provide with new credentials and modify neroprone file, which is rather not really good. In fact, we oppose this approach two the CI and CD. That is to say GitHub's approach, which is a continuous deployment pattern where a deployment agent will take in charge all the deployment step. We at no point describe how to deploy our manifests onto the platform. It is up to the agent to do it, and that's why we say that the GitHub's approach is purely declarative. It will fetch from your sources the different manifest and apply it on the platform, although that it is not linked by definition to the Kubernetes platform. We can say that Kubernetes is well adapted to the GitHub's approach because first, it provides a declarative model through the different resource definition that we describe in YaML file manifest. And second, we have infinite reconciliation loop that these deployment agent can rely on when it will apply the different manifested watch from the repository into the platform. In terms of credential. By design, when we install a deployment agent, we provide it with the credential to access the platform so that we don't have any more the issue of providing into the pipeline the credential. So let's have an overview of Argo CD and Flux. Both are open source and donated to the CNCF respectively by WeWork and intuit. Both are graduated and both have broad community on GitHub in term of stars. Last but not least, we strongly discourage you to install both on Kubernetes cluster. What we may find on the Internet when we compare two tools is that okay, they are complementary and we can take both of them to make the perfect tool. But no, don't do that. It's not a good idea. Let's focus on the model. So what we understand by the model is the different entities and relationship that would be considered by a given tool, flux or Argocd. It is important to have it in mind because it will help you to understand the deep philosophy, the deep understanding of how those tools understand Githubs and will help you later in the usage of these tools and especially in these decision of whether flux or argo. Before going into the description, let's remind you what is an operator pattern on Kubernetes. Kubernetes is a platform that allows you to industrialize the deployment of your application through different entities like pod deployment, job services and so on. Which means that Kubernetes understands its business logic. Business being deploying an application. Now what if you want kubernetes to understand your own logic, your own business logic, for instance githubs. Well, with the operator pattern you are able to define your own custom resource with an associated controller so that each time you create a manifest foo, you will trigger a controller that will later interact with Kubernetes to do this and that. That is two say Kubernetes is now able to understand your own logic. It is interesting to know about this operator pattern because both Flux and ArgocD relies on it. The flux model an overview is this schema, but it has not many sense if we don't know about the different customer resource definitions. So let's go back on this schema later. The first resource to consider is the source. The source is where flux should be monitoring the synchronization. To do so we have several kind of it several types, git repositories, bucket OCI repository, and also ELM charts and repositories. We can describe it through basic manifest with their own API version and own kind. It's quite obvious in term of specification. Here we define URL corresponding to this git repository and tells Flux to cons it at the master branch every five minutes. We have the same kind of consideration for an ELM repository and for the elm shard kind we have this specific sourceref tag which points to another ELM repository here, a repository named Pod info that we can find here, but alone a source is not enough, it only tells Flux where to watch but not what to synchronize. And for this we have the customization which relies on the customized Kubernetes tool to not overcomplicify this presentation. I will only tell that the customization is a way to specify to flux what path to consider in the previous source. So for instance here we have a customization telling Flux that you have to synchronize the entities you will find into the pod git repository under the path customize. Of course we also have resource corresponding to an ELM release which is a wrapper of an elmorelize and you manipulate it through Yaml file which definition is quite basic. You have the specific API version and kind then a spec that will tell you the interval of installation, the version and most important the source reference which is here. And of course you have the values needed to render the chart template in terms of organization of those different custom resource definitions. We have at higher level a source which will be pointed by a customization but also an ELM release. And the customization will tell Flux where to find the different Kubernetes entities and custom definition to synchronize with your cluster. An interesting point is that inside those entities we can find also other sources and you begin to understand that in fact we can create a directed acyclic graph that would represent the entire state of your Kubernetes cluster. Something that we have to watch for is that we can create cycle, and if cycle is created, the customization that is at the origin of these cycle, the synchronization of this synchronization of this customization will fail. We also have the capacity to define explicitly dependencies between same sort of kind. For instance, we can say that a customization two depends on customization one, effort flux. It means that customization one should be installed before customization two. And we have the same kind of relation with the nmolis also. So we are back to our overview and what will happen in normal flow flux flow is that you will push source on your repo source kind which will be pushed on the Kubernetes API by flux. The source will trigger a source controller which will read the specification and see that it has to clone a GitHub repository. The cloning will happen inside the pod under the source controller. Then we also pushed a customization that is referencing these very same sources at a given path. So the customized controller will read these cons repository and explore at the path specified in the customization the different manifest that it would find and each manifest will be applied to the Kubernetes API and a namespace controller will take action, airbag controller take action, and so on. And we can also find a releases that would trigger a name controller which reference a name repository and would be able to fetch the right version and install the right release into the cluster. Let's focus on the ArgoCD model. The ArgoCD model is a very different one from the flux model. In this model we only care about one central CRD called application. The application represents the synchronization part of the GitHub's approach. It's an all in one CRD where you define sources which can be plain YML files, customize Elm or chisonet, but also the target which can be defined as a different cluster than the one where argoCD is installed. In a sense we can say that this model is simpler than flux because we only have one CRD, but because we have much more parameter to play with. We can also say that this model is more complex than the flux one. As usual, we have manifest with its own API version and kind here application these we define a project which is a notion we will see later, but most importantly the source these a git repository with a path to explore in order to synchronize the sources we will find into, and a destination which represents a Kubernetes cluster. Where to make these synchronization wave. On the right side we have an alternative specification where we specify a NAM repository and we want to synchronize a dedicated chart called Argo. Sometimes you may want to specify another kind of sources. You can enhance these application capabilities with config management plugin that would allows you to specify other kind of resources. And this is interesting because you can benefit from the community which provide many ArgoCD plugins. You also have the capabilities to synchronize these synchronization of different applications. That pattern is called app of applications where you define a root application, which sources will point to other application CRD, so that once you synchronize this root application, you will trigger the application of the subservices. An interesting point is synchronization hooks. In fact the synchronization have a lifecycle with phases and each phases can be hooked with a job computation so that for instance you can has a precinct job create a database needed for the sync phases. If you remember, in the last application manifest we has this default project setting which in fact points to an application project resource definition which represent in a way the federation we want to apply on a group of applications. These federation means what kind of permission we would apply on the synchronized resources, but also where we can deploy in term of namespace or cluster or what kind of entities the application under these project are allowed to deploy. So in term of dependency model we have this CRD which is to my opinion secondary, which will point to so a project which will point to application which is a central customerial definition of the ArgoCD model and which represents the synchronization. And an application we have seen can points to other application, it's an app of apps pattern, but in these end an application, a leaf application will point to Kubernetes entities, or ELm charts or Gsonet and so on. So now we understand the model of both tools, let's wrap up our findings. I like to say that Flux and ArgoCd are an answer to the same question, which is how do you implement Githubs on Kubernetes? If you ask someone which is interested and well versed into kubernetes, it will tell you flux, because flux is keep it simple, stupid. It already depends rely on existing entities from the Kubernetes environment. An enrollment release is nothing new, a customization is nothing new. And now that we have a terraform operator also on flex, it's nothing new. On the other hand, we have ArgocD, which provides you with a new model, a kind of logical layer that render kubernetes as a detail of implementation in the GitHub's padding. And this is interesting for people such as developers who are not interested into kubernetes. And also I think it explains why by default Argocd has a user interface because it really needs it in order to express these model in term of application and app project, contrary to flux which already rely which purely rely on Kubernetes entities that we already can handle through knnice in term of user interface. So it is interesting to have it in mind when you want to choose between flux and ArgoCd. Let's conclude our comparison with multi clustering and multi tenancy. First, multi clustering we understand from a GitHub's perspective these multi clustering has the capacity to synchronize application on several clusters while using a single tool application for flux. This is not possible because each cluster we want to synchronize has to be installed with a flux instance, meaning that if we want to deploy application synchronize deployments on cluster one, cluster two, cluster three of all of these three must have a flux installation on them. But we can emulate the multiclustering approach by creating a central flux installation root repository that would contain the installation source of each flux installation. That way those installation would be able to monitor themselves and later we would have inside the same root repository sources and customization that would point to application repo that would later replicate on each cluster. So in a sense we have a control plane which is distributed, but still that we can centrally manage through a flux installation root repository containing the source installation of each flux clusters on Argocd we have the multi clustering approach. By design. We have a single cluster on which Argo is installed and the target of the applications we define could be very well other clusters. My Argo installation has access to multi tenancy. Now let's call a tenant a segregation quantum multitenancy from a GitHub's perspective would be a tenant bonded to a namespace and also to permission to that namespace only from a flux perspective we don't have by design, once again the multitenancy, but we can also emulate it by reusing already existing entities from Kubernetes but also from git. Basically we would create repository, a git repository where only operative would have access two, and inside this repository we would define tenants as namespaces, Erbac, but also sources and customization that points to other repository which contains the infrastructure of a specific application. But this time this tenant repository would only allow a dev team to have access to. So we see that the multitenancy rely on two notions. First one being Kubernetes Rbac and second one being the GitHub or GitLab Rbac. From an ArgoCD perspective, it is already rooted inside the tool, as the app project gives you the necessary tools to create your tenant, in what kind of namespace your application can be deployed on, with what right, what kind of entities kubernetes, entities you can manipulate, and so on. And also we can note that airbag and single sign on is supported by the user interface of argocd. So now we are going to see the user experience which will be presented by Clement. You are live. Thanks ismail. So now that we have seen how Argo and Flux were designed, let's see how this relates to the user experience. We have two kinds of users for GitHub tools. These administrator team in charge of deploying and maintaining the infrastructure, including the GitHub tools, and the user team, mostly composed of developers who are not expected. Two have as deep knowledge on kubernetes on the GitHub tool than the starter team. Before installing or using or even choosing our GitHub tool, we must first talk about different things we need to implement on the GitHub's architecture. We must talk about, for example, whether we want a multi repository or a mono repository for the deployment code. If we want a multi repository, what scope should each repository have? We should also talk about how the code should be put into each repository. For example, do we want multiple folder for GCP, Azure or a single one? We must also talk about who owns the code for the repository and who has the right to modify it. For example, the developers will probably have the right to modify the code in the GitHub repository to deploy on the old cluster. But also these SRE team should have access to this repository in case something went wrong in production and they need to fix it quick. We must also talk about how can we protect the production. By that I mean how do we implement for example, or git flux or git flow? Like if we want to deploy first in these development, then QA, then production. That's something that needs to be discussed before and implemented if you want to, but some protection on the branch. So these issue is that these really need to be discussed before putting a GitHub solution or you may need to refactor these code base in the future. They're not mandatory I would say, but it's still nice to do it before implementing it. So let's begin with flux. A Redmissrazer team wants to install it on these cluster to do so they have two official choices. Either install flux with a CLI which will bootstrap the installation on the cluster and create the initial synchronization between the cluster and these provided root repository provided by you, I mean. Or you can use terraform which do the same thing has in CLI but with terraform which is useful. When for example, you are using managed cluster on AWS or GCP. There also exists a community mshat that will install the core component of flux on your cluster, but will not create any initial synchronization between the cluster and git repository. So like I said, flux manifests will be committed to the root repository. That means that if flux detects, for example, that there is a change between the git repository, only its own installation flux can correct this change by itself. So now that you have installed flux on your cluster and that flux is doing the initial synchronization between its root repository on the cluster, you should know that you can put your manifest on the root repository before installing flux. That means that for example, if you have all your manifests already on your GitHub repository, for example from a production cluster, and you want to put them on a pre production cluster, you can do so really quickly. So we have installed flux on the cluster. We have the initial synchronization ready between git repository or root repository and the cluster. Now we want to add over multiple application on the cluster to do so. It's quite easy as long as you follow the documentation. You have really four custom resources that will Internet you the git repo, the customization, the amplitude and the narrative. And using these four customization custom resources, you will be able to deploy all your application on the cluster quite easily. Like the CRD, they really only do one thing, but they do it really well. There are no option on the CRD to really drastically change the way they function, but you can still do some minor configuration. For example, change the synchronization interval used by your different custom resources. So now we have our flux installation ready on the cluster. We have all application running on the cluster by flux. Let's see what happens if our developer teams want to modify something in the cluster. So to do so, it's quite simple. They really only have to commit a change in git on flux will handle everything else. So if you want to do an update, you write your commit, you merge it in the main branch. For example, influx will detect the change and apply it on the cluster. If the change is not up to your standard, you can revert the change using a git revert, commit it once again on the main branch and the flux will change the modification of your cluster. Now like I said, flux really only does one thing, it's one thing, it is synchronizing. So now we have flux running in the cluster. We have the different developer application also deployed on the cluster using flux. Let's see what happens if the developers want to modify one of the application. So the developer for example created a new version of the application and want to deploy it on these cluster two do so they will create a commit on one of the GitHub's git repository, merge the commit in one of the master branches and flux will detect the changes and apply it to the cluster. So to deploy a change on the cluster release, developer only have one tool to know it's git. They only need to know about a git commit git revert. That's all they need to know. If for a reason or another you want to pose the flux synchronization for a component between git and the cluster. For example, you deployed manually a change on the cluster and you needed to stay that way until you've committed it. Two git well you can use an annotation on the cumulative object of flux. So now that the developer has released its new version on git, he wants to check what is happening in the cluster. Well to do so, flux does not provide any UI out of the box. So these developer will need to have access to the cluster and check what is happening with flux and with its application. If there's an issue with the deployment, he will need to go check on the flux. Custom resources what is happening for example on the customization there may be an issue if one of the Kubernetes object is malformed. If the administrator team has properly configured it, the developers can use Grafana to check what is happening on flux with their application. There also exists web UI developed by withworks that will allow these developer to check what is happening with flux and whether its application is deployed on these cluster. Quite simply this web UI is not included in the box, but you can deploy it quite simply using Elm. So now from these point of view of the administrator, when you want to upgrade a flux installation, the way to do it depends the way you install it. If you install it with a CLI, well there is a command CLI to upgrade your application that is quite simple to use, but you need to remember the arguments you pass to the first command to the flux install command. If you're using terraform, it's quite simple, you just need to upgrade the version of the data source you are using and flux will do the rest. I want to put these little point of intention on the fact we use githubs to upgrade the Githubs tool. It can be dangerous to use GitHub to upgrade the same tool that is monitoring itself because if something were to run wrong in the update, well the tool will not be available to do a revat on itself and you would need to go on the cluster to upgrade it. So if you want to upgrade GitHub using GitHubs, I would recommend to use another GitHub tool to upgrade the first GitHub tools. So now let's see how ago CD works. So our team wants to install Argocd on their cluster. To do so they can use either helm or directly use Kubernetes manifest. It can be installed with Kubestall applied or customized. There is no initial git repository required when you are installing argocd. That means you only install the RSD core components and there is no initial synchronization done. There are three installation type that you can use, either highly available installation type, a nonlivariable insertion type, or a core only installation type that does not include the web UI. I would recommend using the non reliable installation one. So like I said, the OCD will not monitor itself. So that means that if you remove one of AGD components from kinesis it would not auto. So now we have our ArgoCD installation completed on the cluster. Let's see how we can install a different application. So IOCD will only use one custom resource to deploy these object on your cluster. It's these application so the application can deploy everything customize Internet catanet it's nice, but that means that there is a lot of different options you can use depending on your application on what you need to do. That means that the configuration of this custom resource can be quite complex. For example, by default these application will not remove anything from your cluster even if it's removed from git. Or if you remove the application from your cluster, you should also check which wave your application should be. That means whether your application depends on these one or not. So there is a lot of different options to use. But that can be simplified when you're using the web UI provided with AgoCD. Now our ArgoCD cluster is running on the Kubernetes cluster. The different developer applications are running using ArgoCD and the developer wants to upgrade one of its application to do so. He will need to commit his change to one of the monitored kit branches and then depending on how the application is configured on ArgoCD, it will either be automatically applied on these cluster or it will require human interaction. So a button to deploy it. The button will be quite nice for example for production scenario. So now the user has deployed these application on the cluster and he wants to check what is happening. Well, he can use the ArgoCD web UI to see if its application is deployed as he wants or not. It will indicate, for example if the application has issues running, for example if a pod is failing. Now from the point of view of the administrator, if you want to upgrade an ArgoCD cluster, it's quite simple. You can upgrade it using either Elm or customize. You really just need to apply the new version of the ArgoCD manifest and it will be modified as you want. I just want to remind you that when you are applying an upgrade on Weatherflux orgo CD, you should really read the release note to check if there are any breaking changes. So to summarize, ArgoCd is quite easy to install using standard Kubernetes tools. To configure, it is a little bit more complex since its customer source has a lot of options that will modify its behavior to use. From the developer point of view, it's quite simple. The web UI will help a lot in that regard. Also, AGC has a lot of sane options that will prevent it from breaking your cluster. For example, it will not remove anything from your cluster and it will always require an human interaction before applying something on the cluster. For flux, on the other hand, it's a little bit more complex. To install you need two give flux permission to access your repository to commit things in it, etc. To configure. It's simple. Each customer resource does one thing and it does it without modifying its behavior. Two, the usage. I would say it's still simple to use for the users, but it requires your users to be a little more experienced in Kubernetes. As there are no safety ads in flux. Everything committed in git will be deployed in kubernetes and everything kubernetes is in git. So what that means is that if you were to remove manifest in git, it will be removed in kubernetes. All right, so now let's talk performances. We wanted to see how angle CD and flux would handle a medium sized cluster of 200 deployments. These two Android deployments will come from both distinct git repositories and different embraces. That it will really match these standard Kubernetes installs that you can see in the wild. The different point of comparison we will use will be the time to deploy an object on the different processes used in the different lifecycle of the tool. We're talking before installing everything, when we've installed everything and it's monitoring the default object, and when it's removing these object, only the core components will be considered for the benchmark. So we didn't install any other tool than. All right, so our setup we are testing our benchmark on an AWS lacks cluster and everyone of our GitHub tool has run git repository already synced into which we will push our default changes. In terms of installation, ArgoCD has been installed with a non highly available installation, so that means we still have the web UI available. We are using these version two 60 for aquacid. For flux we used the standard installation with CLI and we are using the latest version available for us which is 00:38 these in term of initial resources taken when we did not deploy anything else other than the initial git repository, we can see that they are both nearly identical in term of resources. So first the timelines. We can see that flex before the installation of all our components is not taking a lot of resources. Then when we're installing these components we can see a little peak in both cpu and memory usage and that taken average out to not a lot of resources before the removal. We can see that for flux the components taking the most resources is these Elm controller for COCD, same thing I would say we do not take a lot of resources at the start of ArgoCD then insertion a huge peak we are averaging. But again our cpu and memory memory consumption little peak at the end when we are removing things. Fargo CD the component that is taking the most resources is the application controller. So let's compare our deployment. We can see that as a first synchronization. These ArgoCD will take nearly twice the resources in flux does. The time to deploy all objects is identical for both of them. So it's not here a good comparison point I would say on the day to day operations we can see that now obviously on flux both have reduced their resources usages and now they both have the same point. I would say in term of CPU consumption, but ICOCD now consumes a lot more resources in term of memory than flux does. So this metric is the most important to you I think since this is what will be on your cluster 99% of the time on the production side, we can see that ArgoCD took a little bit more time than flux, but it's not consequential I would say, and that once again ago CD took more resources than flux. So the results we can see that in terms of deployment speed, both ArgoCD and Flux are fast. These will be faster than anything you can code in your CD pipelines. In terms of resources consumption, we can see that ArgoCD is taking a lot more resources than flux. We think it comes from the architectural choices Daco city took in that its application is doing everything its application custom resource. I mean in our experience, this memory consumption of ArgOCD is really impacted by the number of objects it is tracking in Kubernetes. So if you are having a large cluster ago CD will be needing a lot of memory. So we have arrived at these end of our talk. I hope we've given you the necessary point to compare both OCD and flux. So two recap OCD will be better suited for you. If you need to ensure that your users need to use a button, for example to push two production. They don't need a full auto GitHub CD pipeline. If you want to give your users a rich web UI so that they can see what is happening on the Kubernetes cluster under the CD pipelines, ArgOCD is a better tool and if you want to protect your production, by that I mean ensure that only some users can push the changes to the Kubernetes cluster and that these changes will not remove anything from the Kubernetes cluster. ArgoCD is better suited for you. On the other one, if your users are experts in Kubernetes, if they understand that everything that is happening on the git repository will be reproduced to the Kubernetes cluster, and that these git and Kubernetes airbag are good enough for you, well then flux will be better suited for you. We want to end this talk with some encouragement. Two, the whiskey team that helped us a lot to create these talk on these. Mainly Basin Fergie who gave us a lot of advice on how to handle ArgoCD on orchester. I hope you've learned something today. Thanks for listening two.