Hello and thank you for joining me today. Today my talk is on end to end testing formerge and why it's the latest and greatest best practice in continuous integration and continuous deployment. And I'm going to show you how you can do it by leveraging ephemeral environments with your CI. All right, let's get started. So today at Comp 42 DevOps, I wanted to give a quick outline of what I'm going to talk about. I'm going to start off with the three main types of automated testing. Then I'm going to go into how you can get more out of your CI by using it more frequently and testing more often. And then I'm going to introduce the concept of ephemeral environments as a vessel for more frequent testing. So here are the three primary types of automated testing that you'll run into in your CI, and these are all things that you can do in your CI, but the question is how often? So unit tests generally can be run against every commit in PR, just because they're very simple. They usually just give an input and assert that it equals to what's returned. Integration tests are a little bit more complex. They're testing your function against existing functions and ensuring that the results match. But end to end tests, which is what our focus is on today, test entire application workflows, which can be really difficult because they do require a production like copy of the latest changes to your code base, which is why end to end tests often don't get run until your staging environment. That's just often very difficult to do, to spin up an environment on every single branch, and most companies do not have the infrastructure for that. I like to think of CI as more than just a pipeline, also a principle, as opposed to bundled releases. You're having frequent smart commits and maybe deployments every single day. If you're so lucky, you're able to automate your tests and run them on smaller code changes. You can also automate things like builds and limits, and that way you're continuously writing code that's getting merged to your main branch as soon as possible. So I wanted to give a quick overview of what the default end to end testing workflow tends to look like at most organizations. So oftentimes you're working on a branch and you're developing to the point at which you're ready to open a PR at which you'll loop in one of your friends, one of your peers for code review. They'll check that out, they'll give you any feedback, and then you'll keep iterating until it's merged to main and deployed to your staging environment, at which point your end to end tests will be run against your staging environment. Although they might be run against instead of just your own PR and your code changes themselves, they might be run against multiple merged branches from that day, because who knows, you might be only testing maybe twice a week. So once you get back, the QA engineers are going to sift through all the bugs that were uncovered on the staging environment and send you right back to the drawing board, at which point you're going to try to fix those bugs, open a new PR loop in your peer again for code review, and the process will continue until all the bugs are eliminated. And so this makes it really difficult for QA to estimate releases because you don't know what bugs are going to be uncovered and when they'll be fixed. And you oftentimes have several days of waiting in between a merge pull request and a test run. So there is a solution to this and it's testing before merge, and there are many benefits to running your end to end test closer to the PR. You're identifying issues faster, you're keeping bugs closer to the developer, and you're limiting context switching. And it really just enables flow for a developer to be able to run their tests every time they make a commit and have those bugs be identified right then and there, and they can just go back and fix them, make that commit and be absolutely certain before their PR gets merged that it's entirely bug free. And this is what the workflow tends to look like. It's a lot faster, it's much less of a loop and more of the inner loop is entirely for debugging. And the outer loop is pretty straightforward because the only bugs that you tend to run into are your features interacting with other features on the staging environment. But basically you'll be developing. You'll open a pull request which will trigger your CI to run your end to end tests against a fully built ephemeral environment based on your branch that you're working on. And at which point it's important to loop in your peer after the tests have run, because it really shouldn't be their job to uncover any bugs in your code. They're just really checking for best practices. And then at which point if there are any bugs, you've solved them, you've passed the code review. You can merge that to main, deploy it to staging, which again will run your end to end tests one more time against the multiple merge features and then you can deploy to production. And this is a much faster, much faster workflow because it cuts out oftentimes several days of waiting. And once you have your feature, it's pretty much in. You're not constantly iterating and patching it, you feel confident in it. And you've tested before merge. And so in order to do this, I touched on this a little bit earlier, but what you need to do is you need to work with PR environments. And what you'll do is you'll attach a full stack ephemeral environment to every pr, and this is going to be based, built based on the branch that you're working on. And it will be infrastructure identical to your branch, including the database. It's basically going to be a copy of production with a few things simplified, like you'll have less data, but it will be very realistic. And then against this environment, you can run all of your tests. You can run any test that you would against your staging or your production, because you'll be able to run your unit integration and end to end tests against this environment. You'll be able to perform code review. You won't just be looking at the code itself, but you'll be looking at the code side by side with interacting with the environment and seeing how your features perform through human testing. And then what's really nice about these environments is because they're Gitops enabled, their lifecycle is automated, so they will automatically shut down upon a merge. So I just wanted to give a quick demo on using Cypress end to end tests on an ephemeral environment with shipyard in GitHub Action's CI workflow. All right, so here I wanted to show you what it looks like realistically getting your app set up to run end to end tests before merge using an ephemeral environment. So here's my application. You can see that it's built and run using Docker build kit. Can see the front end and backend services here. And I define these services in my docker compose file, which we can look at here under the configure tab. And it parsed out the services right here. We're going to direct our tests towards the front end service, and our back end service has a database attached to it. And right now we have some nvars that are injected during build and runtime. And one of the nvars that we're going to pay attention to is the front end domain for the front end service. And this is where we're going to point our end to end test. So that will come in later. We're going to configure our application that way. I'm just going to show you what it looks like creating an account and using this application. This was developed by the Cypress team. It's basically a Venmo clone, and it's a great way to showcase end to end tests with Cypress. And this one here we're running in an ephemeral environment. So as we can see, this is a Venmo clone. It has a few basic features. It has user settings, you can add multiple bank accounts, it even notifies you when you've been requested to pay. And since this is all fake money, we were just going to make a request for $1,000 for lunch just to show the functionality. So we can see that there are a few main things we can test here, but I've written a few short Cypress end to end tests that will just do some basic functionalities just for this purposes of this demonstration. So I'm just going to go to my terminal and I'm going to run the Cypress tests that I've written and we're going to see them run now. So I configured the app to point the end to end tests at the ephemeral environment URL, which you can see right at the top. So it's going to be the environment that I just showed you. And these tests are super simple. All they do is they just create a username, create an account, sign in with that account, and go to user settings. So years we can see everything's working as it should. And I'm just going to populate these fields with an email and a phone number. And this last part here is just checking to make sure that the save button is of type submit, which will matter later. All right, so as expected, all the tests passed, but I want to take this to the next level and I want to do this entirely in my CI. So I'm running this. So I forked this copy from Cypress and I do have a GitHub actions workflow set up. And what I'm going to do here is I'm going to set this up to automate my end end tests. So right here we're using the shipyard GitHub action, which you can get on GitHub marketplace. And basically what it does is you go to your GitHub actions, environment variables, and you set your shipyard API token and your bypass token. And what it will do is it will take those and it will auth into your ephemeral environment and it will be able to run the end to end tests against the environment. So that's a real sample workflow that we're looking at right now. And it's quite simple. We're just going to use a custom run command because we only do want to run that particular end to end test in this case and that's our custom command there. So that's the exact thing that I ran in my own terminal and it's going to perform exactly the same, except this time it will be automated on every commit and pull request. So here we're going to see this run, but this is going to take a while. So in the meantime I want to create a new feature. So right here I'm just going to change the submit button to a button of type reset just for testing purposes. And we're going to pretend that it's a cool feature. So I'm just going to commit that in a new branch and publish that. And then based on the new branch we're just going to create a new pull request to the base branch of my forked repository. We don't want to accidentally send that to Cyprus's repo and we're just going to pretend that this is a PR for a really cool new feature that we've added that we do want to run end to end tests against because we want to make sure that everything is functioning as it should before we merge it. So very nice thing about this is we have set up our GitHub actions workflow which is going to be triggered upon that PR opening and that will run just like the one that we were previously viewing. It's going to take a while to run, so let's go back and let's see how our initial run of the CI workflow is going. It's been about a minute and a half and we're getting through the workflow. All right, so I just have this sped up a little bit, but what it did was it authenticated into shipyard, it has the necessary environment variables and now it's running end to end tests against the environment. And again, this should function identically to how it did in the terminal because it is just connecting to that environment based on the URL and bypass token. And the first test passed and as expected, all the tests have passed. So that's awesome. And the nice thing about this is now this will happen on every single commit, so that way you can make sure that your new feature branches are working. But let's go back and let's see how our pull request branch is doing so that pull request has built in a new environment. So we're testing against that new environment environment separately from the base environment. So I have this at like eight times speed, just because we don't want to wait through this, but everything is going as it should. We're just about to integrate shipyard. All right, that worked. And we're going to run end to end test against the ephemeral environment. So again, this is my new feature in a PR environment. And I did change something that I specifically changed so it would break the end to end tests, because there is one test that specifically checks that that button right there is of type submit. So it's struggling here. And thank goodness we found this out now before merging that to the staging environment, because we wouldn't want to break staging with failed tests. So I think it's pretty clear here. I think, thankfully, in this case, we know exactly what to fix. But before we do that, we want to go to the PR environment and we want to do a reset here. So it's connected to our base environment and they share the same database, and we want to load a snapshot from the base environment into the PR environment and we want to make sure that we reset it to a previous state because we did create a new user account and we did a few things with the database that we want to kind of clear from our view, just so we can start fresh and be running the end to end tests against the environment with an identical database every single time. So we're just copying that back there and we are going to fix this little bug that was uncovered that broke the end to end test, which is just as simple as changing a button, and we're going to just push that, commit, and just like that, it's going to trigger our CI workflow once again, which I also have at eight times speed. And right there, we're authenticating into shipyard into our environment. All right. And then we have our environment here. All right. And as expected, the first test passed, but now the moment of truth, if the next test passed with our fixed feature. Again, thank you very much for joining me today. I hope you enjoyed this presentation and I hope that you learned something new. And if you have any questions, feel free to email me. Natalie at shipyard build and I'm more than happy to talk about end to end testing, ephemeral environments or anything DevOps related. Always happy to hear questions. I hope you enjoy the rest of comp 42 and have a wonderful day.