Hello there. Welcome to the Conf 42 Rust Lang two conference. My name is Atmaram and I am going to talk on the topic of journey of journey programming language written in rust by tester. None other than me. There are so many programming languages out there, yet I felt need of writing another one just for testing and I am going to talk about all my experiences in writing this language in rust. About me I am an occasional speaker. I talk at multiple conferences. I am blogger. I write lot of blogs about testing and systems engineering. I am passionate programmer. I like writing code, writing good code. I'm tester by profession. I do testing for earning money. I am open source contributor. I occasionally write on Twitter with the Twitter handle atmnk nine. Don't forget to subscribe and I work at Sedin Technologies as test manager. So I am going to walk you through the agenda of this topic. I am going to talk about the idea of journey programming language first. Then I'm going to quickly show you the demo of how journey programming language works with its quick application to sample application. Then I'm going to talk about how it came into existence into various phases, how it started as a pet project in Java, then in between how I felt need of writing the same tool in different programming language. Then I will talk about how I chose the rust for writing this tool. Then I will talk about how I created a project plan and how that project plan worked. Then throughout this journey of writing tool, how different features came into existence, got added and some of the feature got removed. Then I will talk about my experiences with Rust while writing this tool and I will show you an extended demo of the same tool and I will show you some rust code as well where this tool is written and I will conclude my talk. So let's talk about the idea of journey programming language. While I was doing testing, I felt a need of some tool. Why? Because I was doing so many mundane activities and routine activities repetitively, I failed need for automating those repetitive activities. I felt there should be some tool where I can automate these activities as command. To elaborate more on this idea, what I failed was I should be able to quickly create commands for multiple or repetitive use. The command should be parameterized means I should be able to invoke same command with multiple different values while running the command. If the tool doesn't know specific value, the tool should ask invoker of the command. The specific values and these commands could be created for multiple tasks or automating multiple activities like test data creation or creation of mock server. So that was all idea and I will show you how this idea came into existence with a demo. I'm going to show you a quick demo of core compiler which has programming abilities. Program which core compiler requires can be written in journey programming language which is an open source tool. So what I am going to do is I am going to switch to the postman. So I have created one server called as a to do server, our goto application for doing pet projects. Right. This to do server has rest APIs. So if you see the post method in this postman has localhost 9001 as an address. Now if I send a post request with a title bring milk, what happens? It creates a to do with a title bring milk. It gives it an id one and it sets its status as done as false. I can create one more to do bring eggs and it created one more to do. I can get all to dos using get method. So it gave me both the to dos bring milk and bring eggs. I can get a single to do like I need to provide the id of the to do. So I provided id as one and it gave me to do with id one which has title bring milk. So this is a sample application. Now I am going to show you while testing this application you will need to create so many data and all right what you can do with the tool which I have created. So I am going to go into the terminal now. Let's try to create a to do. So what I'm going to do is I'm going to run a command car, run create to do. Now if you see this command ran and this command is stuck at something, it's asking me please enter value for title of type catering so I can say bring vegetables. And it said it has done executing journey. Now let me see on postman whether this to do was created or not. So I'll say get all to dos and it has created a to do bring vegetables. That was the same to do that we created right over here on terminal, bring vegetables. Now to get all to dos I needed to go to postman, right? But same can be done with the command which I have created. What I am going to do is get all to dos. Now if I run this command, it is giving me all the to dos. If you see the response is a string which has to do with title, bring milk, bring eggs and bring vegetables. Now this is not what just the tool can do. I'm going to show you one more command, create multiple to dos. Now I want to create multiple to dos. Now how many to dos I want to create this is what tool is asking. Please enter value of to dos to create length. Now I'm going to give let's say I want to create twelve to dos. Now the tool is asking do you want random to do names? And the value which is asked is of type boolean. Do I want random to do names? Let's say yes and it said it's done executing the journey. Now let's get all to dos. Now you will see all to dos are over here. Whatever to dos these random to dos with random title were created. This is what tool can do. Now you must be wondering that is this tool just coded for to dos? No. In fact get all to dos create to do. These are all scripted. You can create multiple commands like this in the tool, how to script this and all. I'm going to show you in an extended demo. So let's switch back to the presentation. Now I am going to talk about how this idea started. It started with a lazy tester who wanted to automate mundane activities. Now this tester initially started this project with plain Java. Initial pilot was started with the java the tool which you see right now. The same tool was written in Java at the right hand side over here if you see there is a screenshot attached for how you going to create the commands. The entire tool was config driven right hand side. Whatever you see is just a config for creating commands. You needed to create folders and files in specific structure. All control structures were config driven. If you see all the control structures like block post all this info were config driven. It was highly designed on a specific convention that needed to be followed. Like you need to understand the convention with which these configurations to be created. To run a tool you needed to have specific version of Java and tool was bundled as a jar. It was used for real project as a productivity tool. So my past organization, in my past project I have used this Java based tool for creating multiple commands, productivity commands. Now what was the overall feedback? So I spread this tool across my colleagues and they started using this. And what was their feedback? They said the idea is really good tool like this simplifies lot of mundane activities but for creating actual command it is bit difficult and creator need to understand lot of convention followed. Also the tool has a dependency on specific version of Java and user needs to install Java. The tool is restrictive in terms of possibilities that can be done with it. So overall the feedback was this guy over at the right hand side saying then I felt I need to write this tool in a different programming language. Why? Because we need a language which can produce native tools. So I discarded Java. Not to bash Java. Java is useful in many places, but for this tool Java was not useful. Then I charted down what are the different needs from the programming language that I am being to write this tool in. So I said language should be able to produce cross platform executables because I needed a native binaries and I was intending to distribute this tool across Mac, Windows and Debian OS. So I needed a language which can produce cross platform executables. Language should be memory safe since tool is going to be highly programmable because we are going to write so I discarded the approach of config driven commands, rather I took the approach of code driven commands. So we are going to write a programming language or a domain specific language and as it is going to be highly programmable, the language in which we are going to code. This should be memory safe. Language should cater to high concurrency needs since tool can be further adapted for writing mock servers and load test because we are going to write mock servers which could be invoked with high concurrency and load tests with high concurrency. So language should cater to the high concurrency demand ecosystem around. Language should have ready made libraries for creating CLI tools. As you saw in the demo, we were invoking entire tool with the command line arguments, so we needed to create a CLI parsers and all language should be systems programming language since tool may be further extended for lower level programmability. Language should have low or no GC footprint since tool will be further extended for load testing because in load testing every user's interaction matters, right? So it should have low or no GC footprint and language should be typesafe. So based on this, I evaluated three systems programming language, C plus plus, Rust and Go. At that time, both all C plus plus, rust and Go are systems programming language. They can produce native binaries and they can support very high concurrencies and they have libraries for building CLI tools. Everything was same for all, chose three languages up till here. But when it comes to memory safety, memory safety is not a language feature and is programmer's job in case of C. In case of Rust, Rust is a memory safe language and memory safety is forced on a programmer. In case of Go is also memory safe, but it's not forced on a programmer and programmer can choose to escape memory safety. When it comes to C. C has no GC footprint and memory management is pure programmer's job when it comes to rust. It has no GC footprint and rust forces memory management via borrow checker when it comes to go. Go has a little bit of garbage collector which periodically scans heap for unused objects. So when I compared the need with the languages and the three languages, I thought like Rust will be matching most of the needs. I never knew how to write code in rust. I was new to the rust, but I still took that challenge and I started writing this tool in Rust. The lovely rust. Rust is called as the most loud programming language. I have heard this statement about Rust that Rust takes its type safety on steroids. Like you can't escape rust type safety. The code won't even compile if it is not type safe. And there was one exaggerated statement I had about trust that if your code compiles it will less likely have any issues. Like even to get your code into the compile state you need to fight lot with the compiler. That's what I have heard about trust. Although it is an exaggerated statement. Your code may still have functional issues and other issues, but you will have some sort of assurance if your code compiles then I created a project plan for this entire project. I designed the architecture which you can see on the right hand side. All the green boxes over here are the code blocks. All the blue boxes over here are the compilers and tools which I use for creating this project, and the black boxes which you see are the binaries which are produced. The pink box over here is a small plugin that I created, and the red box over here is an intellij idea for which this plugin was created. This entire tool was written in rust programming language, although there was some one more project where I created a plugin for this tool in a Kotlin programming language. But wool tool is written in rust programming language. The plugin which I was talking about is an intellij plugin which can be used with Intellij idea. So that Intellij idea understand this journey programming language and it provides intelligence around its syntaxes and other stuff. So in the project plan I designed and created architecture of this entire project. Writing unit integration and end to end test were necessary for this project because this was going to be an open source project and I may sometime have time or sometime may not have time and if there is a need for any feature then I should be quickly able to tweak the code with getting lot of backlog that I have lost my touch with the code. So I should be quickly run the test to verify whether I am not breaking anything. Creating Intellij idea plugin for adoption of language was also necessary. Adding more features as and when required was also necessary. Evaluating existing open source libraries for not reinventing the wheel like in the entire progress of this writing, this tool, I used so many libraries which I am going to talk about so that I don't need to write so many code, custom code by myself. This was a project plan that I had. I'm talking about the designing and architecture of this project. This project was built into four different components. There were three rust crates that were created which was one of those was server, other was compiler and other one was library and there was separate project for Intellij plugin which was Kotlin Brace project. The server which I called Cors was a websocket based server which can be interacted with. It can be integrated with any messaging application. So the tool which you were seeing, the commands which were invoking on item or command line can be invoked from any messaging application. The compiler or the core compiler that compiles project and creates distributable package which can be used with servers and other was also one of the crate and most meaty part of this code was a library that has most meaty parsing and execution logic which can be reused in server and compiler. And the plugin was an intellij plugin to provide intelligence around syntax of the language and I needed to write unit integration and end to end test as a safety net throughout this project progressed. This project started as a tool for test data creation by invoking rest APIs, but it saw lot of demand for the features. Like one of my employer asked me to add a feature of service virtualization because the product which we were testing had integrations with lot of third party systems and those third party systems were not providing good sandboxes. So we almost automated like ten plus third party systems. We service virtualized those ten plus third party systems with this tool with the added feature of service virtualization. I also added feature of test data creation directly into DB. So you saw I was creating test data using rEst APIs, but I can create test data directly into the DB using the tool as well. Then one of my employer was testing websocket based servers. While I was researching for any ready made tools for load testing websocket based server, there were some tools but they were not handling event loops properly because I used Tokyo in this project which was already having event loop. So I just needed to spawn the scripts in a load test manner. So load test feature also got added into this tool. Then later on I removed cores which is a server component which can be reintroduced later because it had low adoption, many people were not using it. I added CI CD pipeline to automatically create platform specific installables like Homebu, formula, windows installers and debian installers. And all these installers are being used because some of the folks from the teams were using Windows machines, some of the folks from the team were using Linux machine and some of the folks from the team were using Mac machines. Now throughout this all writing journey of this tool I had really good experiences with Rust. I had initial typical struggles of fighting with Rust compiler for code to compile entire async ecosystem in rust is highly customizable. Crates like Tokyo, Asyncrate, Norm, Clap Sud were highly used needed to use nightly since many language features and libraries used require nightly features. Some libraries were either nonexistent or were not in mature state. So I needed to create quick and dirty reusable libraries like async, rdbc, etc. Every time a code that was compiled was rewarding since it used to ensure something working has been reproduced. Writing unit integration and end to end test provided necessary safety net to modify features regularly without being anything so I can carelessly tweak the features and unit integration and e two e test used to give me enough safety net that I am not breaking anything because there were already users using this tool. I got all answers in community forums and on community discord server when was stuck. I used different libraries while coding for this tool. Like a norm for creating parsers and ast I used Tokyo as an asynchronous runtime. I used hypher for low level servers, I used cert and cert JSon for serialization and deserialization. The clap library came handy for cli parser futures and futures utils for some readymade futures features. With async code there is this very good trade called async trade for lot of async code with trades. I use async recursion as well because at some places I needed to write recursive code in async blocks. I use fake library for writing some fake data generators. I use Mokito as well for creating test doubles for test and I use many more other libraries. Now we will see the extended demo of this tool. So you must be wondering like how this code is written, right? So let me show you the code which is creating a to do. This is very small chose which is just eight lines of code. You can see I have a create to do which is parameterized which has title. I am calling post request. So if you see this looks like a programming language, you can see the syntax highlighting as well. Post request I can pass URL text this text is like a text template. I can pass. I can provide variables as well over here with body object title title. Now if you see over here, title is nowhere, nowhere asked for or nowhere coded. So this tool uses a principle of correlation. Whatever is not known it will ask the user. So I'll show you quickly what I mean by that. So when I say chose run create to do what is actually happening is it is looking for title within a code within its existing runtime. While this script is running, it didn't find title so it asked user please enter a value for title of type string. I can say bring tomatoes and it ran the journey. Similarly, when I am calling get all to Dos, you will see I'm calling a get request with URL text like this. Now here you will see there is one more correlation happening matching body object result. So this tool has ability to capture response from the rest APIs as well. So if you see get request is done over here. But this additional clause matching body object result will capture the response in the variable result. And here we are printing text template result. So if you see I have used text template with a variable result. So if I say get all to Dos, let's see what happens. Get all to Dos, you will see this code ran get request. It stored the response in result and it printed the result as a single line. You can see this code is just six lines of code and it can produce tremendous results. Now let's talk about get single to do. Okay? Before going showing you the script, I will just run get single to do. Now it is asking please enter value for Id of type positive integer. Like what is the id of the to do? If I say id is three, it gave me the to do with id three. Now let's see, what is the script written for this? So you'll see it is calling a get request. And as we said like this is a text template where we are passing id as a parameter and we are declaring that the id should be positive integer. So while you saw it asked me please enter value of Id of type positive integer. So let's call the same script and let's see if I provide text instead of integer. It says that invalid value. Please enter value of type positive integer, right? So if I say, let's say ten, it gave me to do with id ten. Now you are seeing all this is happening on a server, right? Like to do server. In fact this entire to do server is also written in the journey programming language. You can see how we can do service virtualization. So you can see here what we are doing is we are listening on 9001 with these scriptlets like we are saying on post with URL. This is again text template matching request body object title. So we are capturing variable title from body object and we are creating a to do object. We are pushing it into the temporary variable to Dos and we are responding with status 20 one and body object to do. So. This is like a dynamic service virtualization that we did for entire to do server. In fact we wrote entire to do server in this programming language. Similarly, when I say get all so I am just returning all the to DoS like respond with status 200 and body object to DoS. When I am asking for single to do. I have this text URL as a parameterized id as a positive integer. So it will automatically grab from the URL the number and it will assign id variable that value. The resultant status is 404 and resultant body is to do not found. If this loop doesn't succeed then this value will be remaining as it is. So you will see on the for loop what we are doing is we are searching for that to DoS for. So this variable is iterated and item Id is compared with this id. Now once this id is found then we are setting response body as item and response status as 200 and we are responding with a status r status and a body object r body. So this is how we did entire service virtualization. In case you are interested to have a look at the code of the server. So this is purely written in rust. As I said, there are three major crates which are written over here. One was deleted, course is deleted. So we have only two now. Chose and Corlin, I have examples over here and playground over here like playground. I created just for testing the scripts actually. But if I say chose, if I say cargo test, it will run all my test. The entire project is test driven and you will see all the code written over here has followed the rest structure. We are writing library in journey states, steps and all. So you have seen like this has passed so many test cases. 263 test cases are passed. So this was all about a presentation. Now I'm being to conclude my talk with this demo. Overall, writing a programming language in rust was a very rewarding experience. The tool created and a programming language created has been used by my past employers and present for service virtualization test data creation load testing, hence it had a very well adoption as well. The whole rust ecosystem is a great place for experiments, tools and projects like this. The feared learning curve with Rust had a minimal impact as the programming settles. With Rust, it feels more rewarding to create great code than initial struggle of fighting with language. No wonder why Rust is called most loud programming language and I consider journey programming language is as one of my best work and I feel really proud about creating something that is useful using Rust. I convey my sincere thanks for free peak and canva for all graphics for this illustration. I am thankful to confortito for opportunity to present and entire rust community for awesome support and ecosystem. Thank you.