Hello everyone, my name is Mitzvinik and I'm a developer advocate on the Facebook open source team. Today I'll talk about modern web testing and how to go beyond using nonselenium when it comes to web. So let's take a look again. Thank you everybody for joining. And as I said, today we'll talk about modern web testing and what do I do? So I give you some of my credentials. My name is Nietzsche. As I said, I'm an open source developer advocate. What it means is that we're working on this Facebook open source program where we are trying to empower diverse communities through open source. Since this screenshot has been taken, we actually updated our website. So I invite you to go check it out at opensource facebook.com and you'll find our videos, blogs and the projects that we are advocating for. I'm also focusing on mobile, in particular Android, iOS and hybrid development like react native. And as I mentioned, some of the projects like lethal that helps you to build declarative UI frameworks, fresco and flipper for debugging on mobile applications. But less about that and more about open source. And again, I'm very much passionate about open source. Hence I'm talking about testing today, testing using open source technologies. What are our goals for today's presentation? I always like to establish goals because they will help us to establish our agenda. And first and foremost, I'd like for us to choose the right test context and know where we're working with. Then we need to choose right level of testing as I'll introduce you. Hopefully you already know of test pyramid and how to actually apply it to web context. And last but not least is choosing the right end to end test approach because that will be the most valuable, I hope. Takeaway from this presentation is how to do end to end testing that satisfies your developer's needs. Whether using Django for development or anything else, we will try to take an approach of an end user regardless of what you are using in the back end or even front end. All right, so let's get to bit more details for this presentation. And how do we usually test? That's an important question to ask. So we're all on the same page here. As I mentioned before, the test pyramid is usually the very good guidance and what people are introduced to. When we are talking about testing first level, the lowest level and the widest of them all, meaning supposed to have the most number of tests unit tests. They are just testing a single function, a single unit of work. They are the cheapest to write and maintain the fastest, but they give you the lowest level of confidence in your overall application system level. Then an integration tests take place. This is where you testing multiple functions in play, multiple components, how they interact with one another. And last but not least is end to end test, going through the users scenario, your end user scenario that they will go through while using your application. And they are usually the most expensive ones, the slowest, but they give you the highest level of confidence. Hence you're supposed to use very few of them compared to other levels. But how do we usually test web real test pyramids? I like the test pyramid as I just showed you. In ideal scenario, reality is very much different. What people usually end up using is ice cream cone, pattern, hourglass or cupcake. So ice cream cone, the ice cream cone, it's basically inverted pyramid where you actually have the fewest number of unit tests. Then you have service or integration tests, more of those, and a large suite of end to end tests. I actually worked at the companies before where this would be the reality, and that's usually the reality for many. And on top of this cone you would have manual testing, which again is not very much scalable for large teams and large applications. Hourglass is similar. Again, you have actually lots of unit tests, lots of UI tests, but very few service tests. We can argue whether it's good or bad, but that's reality for many. And the cupcake, cupcake is very much like these ice cone. Again, it's very much like an inverted pyramid. You have fewer unit tests, more integration tests, lots of automation UI tests, and a massive number of manual testing. Again, it's very much like these ice cream cone. But the idea is it's an inverted pyramid, meaning that the pyramid that we all like to talk about is not the reality for most. So we need to change that. We need to change these antipathons that some people would call it. I call it a reality, but I always like to hope for the best and advise us to change that. And the way to change it is first to actually test something. And what is that something? Throughout today's implementations we'll have this simple app to do app that usually people use for lots of different UI frameworks. You can find an example of to do with angular, with react, with swelt, any number of projects, any number of frameworks. And this is a good app for us to test. It's basically you adding these to do, you checkmark it, you delete it, you filter items by completed or active, et cetera. So back to testing. Now I introduce you what we're going to test throughout today. And now let's actually see what we're going to do. There are a couple of scenarios I want to introduce here and go through for today's presentation. These first case would be imagine if you have a backend using Java and for many testing would be Java. In case of Python. Some people use Python, these backend, they will also use Python for testing. I would say that there is a problem with this approach where you try to match what in the back end for actual resonating, for the language that you use for testing, because ultimately for web, for web apps, you're seeing it from the end user perspective. Even if you're writing end to end tests, even if you're writing unit tests, it doesn't matter, you don't have to match your back end with a testing code. And so that's why I'm suggesting us to move away from been bound to whatever is in the back end and just go away from it and rather focus on hybrid model, domain driven development and user centric testing. When I'm saying these words, what I mean by hybrid model is that many companies in the past couple of years, they went through transformation where they used to have testers as a separate team from developers. So developers would write some production code, they would throw it at testers, testers will test it, give the feedback back to developers and there will be like a fairly massive feedback loop, fairly slow one. People have been trying to, companies been trying to avoid that and they called everybody now software engineer. And using this hybrid model where developers are the ones who test and testers are the ones who develop, there is no separation anymore. It means that you are trying to use the software engineering principles, even with the test infrastructure, but you are emphasizing the test expertise that testers bring. It's not just the fact that testers don't do anything and developers just do the important work. No, both parts can actually bring something important to the table and we need to find both of best of both worlds. This is what hybrid model is. And by again going away from a back end testing and just focusing again on the end users, this is why hybrid model is so important. This is domain driven development principle that plays these role here as well. Domain driven design, one of the main ideas these is ubiquitous language. You're trying to speak the same language with your developers as you would with your end users. When a user talks about an account and in the back end you're calling it user Java or something else. There is this misalignment, there is that miscommunication that happens. You want to actually match what your end user would care, because nobody cares what kind of great pattern you use at the back end. Ultimately it all ends up of what your customer gets. So customer is what we care about really here, and we're trying to speak the same language as they. That's why instead of back end resonating, we do front end testing even in the lowest levels, because context matters. You don't want to do this context switching of how you test internally when you talk to the user. And again, it's all about keeping it user centric. And I've been repeating that over and over again throughout these couple of slides. If you're still not convinced why I'm saying that making sure that you match your back end language with your testing language, modern web testing focus on the web. It brings up the power of JavaScript, and I'll be emphasizing that regardless of what you have in the back end, throughout your application, throughout the pyramid, unit integration and end to end, you can use JavaScript because that's what web is for many of us is based on. It can be PHP, it can be something else, it can be hack, but really javascript for majority of the web outside of like WordPress and such. And that's what I will be focusing on today. When it comes to power of javascript, I'll talk about assertion libraries, process libraries and enterprise libraries. So assertion libraries, things like jest, Jasmine process libraries, it's behavioral driven development libraries like cucumber and enterprise libraries like applitools for testing it help us with visual testing. And speaking of language of the web, JavaScript in particular I'll talk about node JS. So when it comes to node JS, it's isomorphic. It basically helps you to write code for server side and the front end for the UI. Then it's fairly flexible, obviously, and it's customizable. And it all comes down to the power of NPM manager as well. You have packages, so you can basically reuse something that other people built and simply import those packages and use them, whether it's for testing or development code. But going back to the pyramid web test pyramid, it's very much the same as any other pyramid would be. It's with unit test, low cost, high speed, and still quite a low level of continue in the overall app. But for the unit testing, for the web unit testing, we'll use Jasmine for an example to showcase how it can be done for your app. So let's say for our to do app, we have this simple function that takes an input, it converts it to JSON and basically returns name and state here, name of the to do and the state active in a completed, et cetera. When it comes to actually writing test with Jasmine, we first initialize the test suite create to do item. These. We create a test case where we accept valid test data. We initialize the to do item here. We basically pass some data to test some data that we know of, brainstorm ideas, and these state is incomplete and we do simple validation. We make sure the name matches what we expected after the app, process it, after the function. In this case, process it. That's what the unit test is. You're testing a single function, create to do item, and Jasmine lets you do that. And I didn't care how that function is processed on the backend. What kind of services are being invoked with what, what kind of language you have at the back end. Ultimately your end user wouldn't call the service directly. Your user would call this function on these front end, on the Javascript side. And that's what I'm going to test here. If I go up the level of the pyramid in integration test in the web context, I would use jest. Amazing framework. It's a test framework. They have so many things there. I'll give you one particular thing called snapshot testing with jest. It's a Facebook open source projects, by the way. So imagine you have this react component that's kind of an old pattern of writing it. But imagine that we do, we have these create to do function, create to do component. I would say that has a render function that returns you a button and it generates a certain action on click and has the label create. And it also has some sort of styling attached to it. I know people usually don't combine component and styling, but this is for simplicity reasons, just to show you visually what a component might look like, in case you've never seen react before, which is totally fine. And let's say if I were to test with just, I would create a test suite, I would say create a to do button is what I'm trying to test here. I render a component, basically I load it, I initialize it, and I call it by what it is create to do, and I convert it to JSon and I save it as we call a snapshot. What snapshot is, is basically this piece of code. It renders a component, it renders everything in that component has the web would see it the function, you can see it's undefined because it's in solutions. Here. Name the class names and styles and if there was anything to be changed to this component, we would catch this after we validate these snapshots. So to reflect on these two levels of the pyramid that we just looked at, every step has only one focus and it's developers. Even though I kept saying that user centric testing is important, I keep talking about those lower levels of testing and their focus is still on developers. Good for some tests, this kind of focus, but not for end to end tests. For end to end tests, we still very much care about our users. And when it comes to end to end tests, just to remind they are the highest cost, lowest speed, but they give you the highest confidence level. And these kind of three points have to be kept in mind throughout your writing tests. So to shift focus from developers towards your users is what we need to do when we are writing this end to end test. So it might sound great to do this shift of the focus, but how do we actually do that? So end to end task testing is what we need to do. User centric testing, it means that instead of interactions, we do tasks. We focus on tasks in hand. You do not care about. For instance, we have a workflow of the user logging in. You don't want to describe your test as user types in, sends keys in for the username, for the password. A user presses the button to submit this form. You don't want that language to be used when you're writing your tests. Instead, user enters username, users enters a password, user presses ok button. You do not go deep into like send keys. If you talk selenium, you don't focus on individual interactions, you focus on the task at hand instead of. If you're talking about particularly writing code, writing test code, these page object model, we're using something called screenplay model, screenplay pattern, where really it's how you write, how you speak, how you name your functions, what matters. But back to our apps, end to end testing is a common way to do. What's the common way to do end to end? Obviously it's nonselenium webdriver. It's at this point, it's a de facto end to end tool. Great tool. It's basically a testing standard at this point. And it's generic, meaning it can be fit for most use cases, I would say. And more importantly, it has quite a high integration with plenty of other open source projects, plugins, weights, objects, et cetera. So it has a great community behind it's an open source project, so send your webdriver. What it actually looks like behind the scene doesn't really matter to us as much, but ultimately what we care about. It has different bindings for different programming language, so what you'd like to use, you can use. But again, I want to go away from matching your back end with your front end testing, and also it has drivers for Internet explorers, Firefox and whatever you'd like. So since I've been talking about JavaScript, in particular the front end testing for the front end, I'd like to bring up Webdriver JS. And by the way, it is important how I capitalize GS because Webdriver Js, with two capital J and S is a different framework altogether. Capital G and J and small s is the official webdriver bindings for JavaScript. So nonselenium Webdriver Js, it's still de facto end to end tool, still a testing standard, still generic, and has a great integration with the open source community. And in terms of how it actually looks behind the scene, just the bindings are node JS exactly what we want here, and that's what I'll showcase here. So we have a suite initialized we name create to do we initialize the Webdriver? Doesn't matter what exactly it does, just believe that it does initialize it. Then you create an actual test case. You're using the valid data to create a to do. You make sure you wait for the page to load, then you send the name of the webdriver of the to do of the new to do, and you basically what is in this case, you make sure it actually appears. And that's about all. You made sure that it appeared on the web. You edit a new to do and that's all. Nothing special there. That's how you would do it with any other Webdriver bindings, Java, C sharp, whatever else. Unfortunately, they're not just these good parts to Webdriver JS. With any end to end test complexities, there is always an issue how you select, how you find your elements on these web, how you locate them, what's these test flow like, and what kind of single page application, what frameworks it can work for. What about Webdriver JS though? Webdriver JS is great, I mean, selector wise it has everything standard id and CSS and xpath bindings. For locators it has by allocators like by dot for test flows. It used to have something called promise manager, because node JS by default is async. You wouldn't want to have your tester or your developers worry about managing this asynchronous. And so the promise manager allowed you to write this kind of sequential code before, but we since removed it and now it's await and async. By we I mean open source community did. And it's still very generic so it can work with anything. But still it looks limited though. I mean the byte selectors are great, id CSS selectors are nice, but is there anything better? And for special case number two, let's say we have angular as a front end. Angular is quite popular. And when it comes to angular, if you've never seen it before, which is fine for the way how it's written, you don't have just regular classes, you have special attributes like entry, submit, model change, et cetera. Basically it helps you quite heavily to write front end. And in terms of actually testing angular, we still use the same pyramid for unit testing, for integration. But for end to end testing we have something special. It's not just Webdriver JS. We don't have to use just Webdriver JS for that. We have protractor. Protractor is end to end test framework for angular apps. It's created and maintained by the angular JS folks. So protractor is basically everything these Webdriver JS has. But then it has additional things, has still nonselenium server, it has API for angular in particular, but also it has connection to the angular JS app. It knows exactly when the app is loaded and ready to be tested, which is a great addition to any these infrastructure. For the protractor selector wise. It has element bindings, it has biri beaters. In terms of the test flow, it knows, as I said, exactly when the app is ready. You don't have to wait for a particular element to appear in order to test, which is a great help. And it's intentionally built for angular. In terms of tests similar tests create to do validate data. You load the browser for a particular URL, you find element by model, you send keys and you make sure it appears. You can see that how much less code I had to write, but I did ultimately the same thing and I didn't have to wait for the app to load. All of that is handled for me. But angular is great. But there are so many other UI frameworks. Obviously there is react, there is vue, there is amber backbone, swelt and many others. I think there are more UI frameworks that I can ever count. But let's say we were to use protractor with react. You technically can, you can remove this ignore synchronization property. It's called. Basically it doesn't wait for angular app to load because it won't ever load with react. Right? But still, it's kind of silly. You're trying to fit one thing into another that wasn't built for it really. I would say non selenium UI test frameworks is the next thing to try other than general Webdriver for JavaScript. And in here, I'd like to showcase two things, test cafe and Cypressio. So what have we learned so far though? Patterns. That patterns matter the most when it comes to testing. The common patterns are how you're handling weights, how you execute in parallel your tests, rapid test development how quickly can I get my feedback from the test as a developer, especially in hybrid development teams? And can I record and do I have an ide for my test automation framework? In terms of test cafe, it handles weights pretty well. It does allow you to parallel execute your tests. It has a great feedback loop, it gives you feedback fairly quickly, and it has a recorder and ide for you to try. To give you an example, if you were to write code itself for test cafe, you can you create a fixture what you're trying to test, you load the page, you can see this stream kind of API style of writing code. You initialize your test, you type a name, you click the button, you submit it and you're done. I'm not going to dive into particular code that Testcafe use, but really the recorder that it has is quite nice. It allows you to kind of, if you have a manual tester team at your company, you can use them to basically record a bunch of these tests and then convert them to code. It's kind of a great first step to teach them how to write automation, and it's also quite a nice way to debug your tests if you wanted to. So if anything, the idea for Testcafe is quite nice. But cypressio is actually more, I think, getting more traction online lately. And the same thing, it handles weights very well, I can attest to that. It has a great parallel execution support, it gives you feedback extremely quickly, and it does allow you to record your tests. It has a special plugin for that. So if you were to write test code for cypress for, let's say a login page, something different from the to do, you basically create these test case for that you specify username and password, you can even tweak cookies right here if you wanted to. You can say you can get cookies, validate those, you can do basically anything you want. And Cypressio has evolved largely from when I first made this snapshot for the code for this presentation and the Cypress. Again, it allows you to record things fairly well. You can see in these browser on the site how it's executed and how you can repeat that too. You can also pause it if you want to debug your tests. And yes, Cypress community has been growing rapidly, has lots of resources about it. So important thing to remember is to choose the right tool for the right problem. Don't try to fit a thing into a use case that doesn't work. There are so many test frameworks that I haven't even touched. Webdriver, IO, Nightwatch, JS, WD, Nemo so many avoid tool mix ups. If anything, do not have different tools test tools in your test infrastructure, it increases the complexities. When you're trying to test a complex thing with a complex test infrastructure, you're making it twice as hard. At least twice has hard there is no ubiquitous language, there is no domain driven design anymore, and there is no common language. So how do you choose what to use for your infrastructure? You can look at the open source projects at the GitHub pages, GitHub stars, how many npm downloads it has, external integrations, and also give it a try. And do the proof of concept first before you transition everything you have to use this framework. But more importantly, choose flexibility and your use case. Build it for your use case. In terms of flexibility, what I mean by that is look at cost transition. Let's say today you're using angular and you use protractor. But what if you know that your team, your company is traditional to react in a couple of months, in a year and two? Or maybe you haven't actually thought of that, but it will happen. So maybe investing in protractor is not the best option. So think of that. Think about return on investment. Does it even worth switching to Cyprus? If you're fully on selenium, it might count hours, count time you spent on maintaining these selenium tests, and then see if it justifies you spending time learning cypress, moving to cypressio, etc. Can you actually customize the framework you're using for your use case if it changes? And can you replace it easily in terms of again, thinking of your particular application, what's your team expertise? If you're all non Javascript developers, maybe again, everything I said in this presentation of testing with Javascript is nonsense for your team. It doesn't worth the trouble to then write Javascript tests. But if you are like a full stack has I would say sometimes team then it might make more sense. Think of what the application uses again, that's why it's important if you're using angular. If you users in react it will play a major role of what these infrastructure you have because test infrastructure is what matters. Can you actually support? Do you care about parallel execution at all? Do your service can handle that? And I always like to end my talks with call for action. Evaluate your current test infrastructure test architecture. Does it worth changing it? Have domain boundaries? Know exactly where you're working? Are you going all the way to end to end tests integration tests? Or you can simply stay at unit tests and then don't bother of my changes these unify your test strategy. Think about it the long run and again don't have the shifts between we have end to end tests with one framework, unit test with another, and integration. We use some old fashioned language for that, so think about that as well. Have it as few hoops you have to jump through when you're testing and debugging those tests as possible. Think of onboarding someone new for your test infrastructure and gives you kind of a good perspective. But experiment first. Don't just convince your team to switch before you do the proof of concept. Give it a try first and then see if it works. So thank you so much. Unfortunately we can't do q a over the Internet here, but if you want to contact me, please do on my twitter, my blog, website, LinkedIn, or directly email me.