Conf42 DevOps 2024 - Online

GitHub as a Platform Engineering Platform

Video size:

Abstract

Revolutionize and accelerate your deployment process effortlessly! Our GitHub Actions-powered solution ensures swift and foolproof code deployment in minutes. No IaC expertise needed!

Summary

  • We're here to explore the role of GitHub as a platform engineer platform. We'll take you from the traditional code DevOps world, through the collaborative DevOps environments to the forward looking platform engineering world.
  • In order to produce what we needed in less time, we created a common code for all of our actions. This allows any user from a dev to a DevOps to assist admin to freely deploy and automate the resources with little to no interventions in just minutes instead of days.

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Thank you for joining us today. We're here to explore the role of GitHub as a platform engineer platform. We'll take you from the traditional code DevOps world, through the collaborative DevOps environments to the forward looking platform engineering world. Today, we will show you how GitHub is not just a repository, but a key factor in building robust, scalable and efficient platform engineering solutions through the use of GitHub actions let me begin with a small story. Not too long ago, the CEO of the company approached both my boss and me, asking us to set up a VM for him. This task took some time, and during the process he asked about the possibility of automating this process. He expressed the desire not to rely on us every time he needed to update the code or deploy a new vm, as most of the times we were busy with other projects. In response to his inquiry, we developed a solution and today I will be presenting it to you. Depending on the size of the company, your infrastructure needs can be met by the project's own developers, a team of sysadmins or a team of DevOps engineers. I will illustrate the evolution of each approach, guiding you from a code DevOps world to a platform engineer environment swiftly. All of this will be achieved while ensuring that everything is checked into GitHub. Not too long ago, companies heavily relied on sysadmins, who are responsible for deploying and maintaining everything. Typically, this is admins employed scripts, repositories with key files, config files, and documentation to set up requested resources. This manual process consume a significant amount of time involving handson configuration, going through documentation, and so on. We are all aware that everything we do in a manual way often led to problems or omitted details. With incorporation of cloud providers, certain tasks became easier, but spinning up a VM and configuring it often involved clicking options and going through a web console. This consumed a lot of time and was prone to errors. The evolution of sysadmins bought us to DevOps engineers and DevOps to the adoption of infrastructure as code tools such as terraform and ansible. This transition improved the establishment of standards and the automation of repetitive tasks through code implementation. However, managing the code, configuring variables, and adjusting settings for specific scenarios proved to be time consuming. While these tools accelerated repetitive deployments, they weren't easy to adjust for each environment. Most of the times, customization was required after finished initial deployments required time to set up, but subsequent repetition streamlined the process, making newer deployments quicker and maintenance more straightforward. Nevertheless, even with the addition of automated pipelines and configuration management tools. Some manual intervention remain necessary. After performing these repetitive tasks across numerous repositories, we developed a tool to automate this process, minimizing manual intervention and saving time. This open source tool we created aimed to eliminate as much manual effort as possible while offering flexibility to adapt to individual user requirements. To achieve this, we integrated popular infrastructure as code tools, incorporated automation scripts, included pre and post hooks to execute scripts, and encapsulated everything within a docker container to eliminate the need to install anything, reducing compatibility issues. The result is speedops, a tool designed to automate deployments adjustable to specific needs, maintaining simplicity throughout the process. However, there were still some requirements, such as defining config files and structuring the repository in a certain way. After that was done, it required pulling the code and executing it with a docker engine. This is where GitHub actions played a crucial role for us. We incorporated scripts into our actions to generate the necessary elements for Bitovi to work out of the box, allowing customization through the workflow inputs. This approach effectively separated the end user from the code while maintaining transparency for those curious enough to inspect it. Furthermore, you will find that within our actions, we provide a variable for obtaining an artifact containing the ansible and the terraform code. The beauty of it all is that everything you perform happens within your repository, ensuring traceability while keeping your repository free of unwanted artifacts like scripts and terraform code. And this is how GitHub transformed into a platform engineering platform for us. And we are here to share this journey and of course the CEO satisfaction with you. However, if you're like me, you might be wondering if Bitovi is essential for this setup. The answer is no. It plays a pivotal role in our story and for many of our actions, but it's not mandatory. In fact, some of our top GitHub actions don't use it at all. What we want to share the most in this talk is the experience we had transitioning to everything GitHub action and the tools we created during this growth so that you can leverage on a knowledge to get you to a platform engineering world in little to no time. But in the end, it's all about the end user experience. From the DevOps engineer perspective, the action yaml defines the inputs and outputs the action will take and generate. The action specifies the steps to run and the logic behind it. Just like a regular script, each step could involve executing a command, running a script, or even calling another action. This provides a wrapper for the infrastructure as code tools and an isolation layer for the developer, it only exposes the inputs and outputs you define. If you're familiar with terraform, this might ring a bell. It looks a lot like handling inputs and outputs in a terraform module. Shifting to a developer perspective, in your repo, you could have one or multiple workflows already created based on your needs. You might be running some syntax checks or some code matrix checks. Adding one workflow to deploy your code to an easy to instance is as easy as adding another workflow and setting the correct inputs. You know that you could trigger all of them at the same time. You could create subsequent steps based on results. The possibilities with GitHub actions are endless. Now, taking the broader view, the platform engineer can outline the entire necessary infrastructure using various actions. In the illustrated diagram, we present an example of an environment based strategy. In this particular case, there are three repositories, each corresponding to a distinct environment. Say you have a dev environment. In your dev environment, you can designate a workflow for every resource that you need, providing a clear and an organized approach to managing infrastructure for all of your environment. You can then copy that complete environment, adjust the required variables, and set up another environment in just minutes. That way, you could have the same things in development, QA, and production really easily. These workflows can be configured with manual triggers, scheduled runs, et cetera. You could, of course, take another approach. This is just an example of how we achieved it. In the upcoming slides, I'll show you some example of our actions and demonstrate how simple it is to deploy various components by setting up a workflow. As you can see from the screenshots, deploying your project to GitHub pages takes only a few lines of code. We try to make the dev experience as simple as possible with our actions. If some adjustments should be made, you might find what you're looking for in one of the many exposed inputs. Digging deeper into the storybook to GitHub pages actions, you will see that the main goal we had was to simplify and reduce the steps into a single line, wrapping up some tasks that we did repeatedly and assuming some default values in some cases. For some developers, knowing all of these steps was really tough and really complicated. One of our goals as platform engineers was to automate these tasks with simple solutions. By the way, if you're interested in more details about how to deploy storybook to GitHub pages, please check their official documentation. Now let's shift to a more interesting and complex deployment. One of the things we didn't like about reusing terraform code was the fact that we had to manually set a lot of variables each time, so we needed something more dynamic. We solved the dynamic part by using the repository specifics, say the repo name or branch, as inputs for the variables. We kept the rest with some same defaults. That way you can get your resource up and running with little to no intervention in just a few minutes. But let's say you want a different instance size. If you want to add a proxy to a database, expose a port, et cetera, you can easily change that by adjusting just an input variable. We went through the process of creating one action for each particular needed we had during our work assignments, but in order to produce what we needed in less time, we created a common code for all of our actions. That way we reused a lot of code and reduced the time to add new ensures. This is a summary of what we produce in just a few months of work. This allows any user from a dev to a DevOps to assist admin to freely deploy and automate the resources with little to no interventions in just minutes instead of days. This is what we wanted to show as platform engineers how we simplified and accelerated deployments of on enhancing the end user experience. That's it. Thanks for watching.
...

Leo Diaz Longhi

Platform Engineer @ Bitovi



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways