Hello. Welcome to my presentation where you will learn how you can be one step ahead of the competition using AWS step functions my name is Oscar Neumann and I'm a solutions architect at AWS, and I want to walk you through how you can use step functions to build modern applications in the cloud. But let's talk about serverless first. Why would you want to invest in a serverless application? Well, I think this quote from Werner Vogel sums it up pretty nicely. Your design time budget is finite, so any time you can spend on writing business logic is obviously a win for you. And serverless services help you a lot with that. But there's much more to serverless than just AWS lambda. We have serverless services at all layers of the application. So for example, we have Fargate as a compute service, we have s three as a data store, and many other services that surround lambda, for example for messaging, orchestration, storage and compute. And we see that when these services come together, you really start to gain all the advantages. But now let's look a bit at orchestration. Why would we need orchestration in a modern app? Well, obviously you know that lambdas are stateless functions that exist in the cloud, but there aren't many apps with only one function, one entry point, one module, or one component. So there's more than one function. In fact, there will be lots of functions usually, and they talk to each other. And in fact applications, serverless or not, they also have databases, right? So the functions also talk to a database. And in the cloud, I noticed that a lot of them also have queues of one kind or another. So they also talk to queues. And some of the lambda functions might also connect to servers. And this is now what an actual modern serverless app might look like. And you notice that these arrows also have different colors, because there are lots of different ways for functions and serverless components to talk with each other and control each other or orchestrate each other. So let's talk about what we can do with functions when we have more than one. So maybe I want to sequence two tasks, so a comes before b, or I want to retry failed tasks, or I want to maybe catch an event or catch an error message, or I want to execute functions or tasks concurrently. Or I want to do a choice based on the outcome of one functions, or I want to run different tasks in parallel. So how could we do that? Well, you could coordinate by a method call and just link the functions together. That's not terrible, but that doesn't give you like modular, independent functions that do one thing well, you could also chain the functions after each other. So yeah, one function invokes the other and you do it synchronously, but, well, that doesn't really scale that good because it might want to call another and so on. So you do it asynchronously, which is maybe a better design, but then you get really hard. Error handling and finding out about errors even requires extra assembly. What you could also do is coordinate by a database so you can track the state of your function and write it into a database like DynamoDB or other relational databases. And you could use SQL for transactions. People are using this and they get good results. But once again, there's still a lot of work. What also could work is the coordination by queues, which is another approach to pass your state and control it through queues or streams. Once again, this involves lot of bookkeeping. The function that reads the queue gets a failure and you need to catch and rewrite the state back into queue. And if you want a timeout or retry, then you're going to have to do that yourself. So what's the solution now? How do you orchestrate these serverless applications? Well, business workflows are rarely sequential from start to finish, so you have multiple decision paths and multiple steps for processing and need to handle failure. So you need to isolate tasks into logical components and centralize the orchestration into workflow. And that's where AWs dev function comes in to help you. So what's AWS dev functions? Well, first of all, it's a serverless workflow orchestration service that's offered by AWS. A workflow in step functions is built using a state machine. It is composed of steps which we call states, and it moves from one state to another via state transition. It's written using our Amazon states language or ASL, which you can think of like assembly for the workflow, and you can use it to orchestrate multiple AWS services. But how do you now build these workflows? Well, there are multiple ways to do it. You could do it using the drag and drop a workflow studio editor. You could just write your own JSON or do it using CDK or DSDK in Python. Then you can visualize your workflow and finally execute and monitor it. Now let's look at an example. So I have this example image processing workflow where I want to first make a thumbnail, then identify features in the image and store the image metadata. For example, I have this picture of mountains, which also contains people and snow and I want to identify that. So how would a state machine look like for that? So we could use different state types for that. First we have the start which is the entry point for this workflow. Then we have maybe a lambda function that extracts the image metadata. Then we could do a type check based on the image extraction metadata. If the image is not supported, we fail this workflow. If it's supported, we store the metadata and then go into a parallel execution we call recognition to recognize what is in the image and to generate a thumbnail. Then we add the text once recognition is complex and end this workflow, let's look at the state language. How do you write these states in JSon? Here's an example how to do that. And there's also really good documentation and this online builder where you can drag and drop and edit these. So you don't really need to know all the document, all the annotations for that. So let's look at some state types. Youll have seen most of these in the talk already, but there are also a few additional ones. For example like the map state with which you can process an array using the state machine. Also there are other helping states like paths where you can transform inputs into another output and feed it into the next state. But what services do integrate into step functions? Well, there are a bunch of services that integrates into step functions and there are multiple types of integrations available. So let's look at the triggers. First, you can trigger the step function workflow, for example using a lambda function. You can also directly trigger it with API gateway or using an event and eventbridge or using another step functions workflow. Also you can trigger it from a code pipeline or using IoT rules from looking at the step function inside. There are also different integration types. There are some optimized integrations which provide additional functionality and UI elements in workflow studio. So you can directly integrate some AWS services, but there are no performance benefits over SDK integrations, it's just optimized. Then there are SDK integrations which allow you to call over 200 AWS services directly from within the step function. And yeah, basically you still need to write less code because you don't need to write a lambda function to call this API. You can do it directly in your serverless workflow. Then there are different integration patterns. So for example request response where step functions will wait for the HTTP response and then progress to the next state. And also you can run a job synchronously so you can call service and have step functions wait for the job to complete. The same is true for waiting for callback. So you can basically asynchronously call a step and then wait for the task token. And once the token is returned with the payload, it continues. So now let's take a look how you can handle and modify data within a workflow. In step functions. So you have input and output processing. So as a step function receive AWS input. It's usually in the JSON format and it passes this input to the first state in the workflow. Then the individual states receive the JSON as input from the previous state and pass JSON as output to the next state here. Then the output of the workflow's last state becomes the final execution result. ASL now has different tools to manipulate these inputs between the states. So let's look at the anatomy of a state and what you can do to modify this input. So of course, first you have the state input, and this state input is passed to the task. Now you can start to modify this input in the task already before executing the task. So first there's the input path and all these paths. They use the JSON path syntax to modify and select a portion from the JSON input or from the result. And the path is a string that begins with dollar sign and identifies nodes within the JSON text. So with the input path you can select a portion of the state's input. Then you have the parameters field, which allows you to create a collection of key value pairs that are passed as an input. Then you can call the AWS service integration, such AWS, a lambda function invocation, or an activity worker. And you can filter these using the JSON path expression AWS the input again. Then you get the result selector. So you can now use the result from the integration and filter from that again, and then you can put that result back into your state input using the result path. You can either overwrite the input at all or just append to it in a certain node. Then you have the output path after the final result and that becomes the state output. Finally. So let's look at the different states, which I just mentioned in detail again. So first we have the input path. As I mentioned, it allows you to filter and select the portion of this JSON state input to use. And yeah, here's an example where we select, for example, the numbers field three, four using dollar sign numbers. Then we have the items path. That's a field in the map state which allows you to select an array in the input. So here we select words as an example. Then we have parameters, the parameters field creates again the collection of key value pairs that's passed as an input to the service integration. So for example, we can add static values here like just a string, or we select the first or the second number from an array as an example. Then we have the result selector. The result selector allows you to filter and construct a new JSON object using the task result. So for example, we have the task output which is sum seven, and we select the sum field and add a static value. And then we have the result. Then we have the result path. As I mentioned, you can now add this result to your JSON node from the input from the original input, or you can also override it. Then we have the output path where you can filter the final result before it becomes a state output again. Yes. So let's look at the context object. You can also get some context information from the step function's execution outside of the state input using dollar dollar execution. And then you can for example get the id from the execution or the start time, which is useful in many cases. Just that youre heard of it already. Then there are also some intrinsic functions which you can use to modify the inputs in the state already. So for example, you can format a string, you convert it from escaped string to jSon, or from json to string and do multiple things. There are also some more intrinsic functions which are not listed here. So now let's go to the different workflow types of a step function. One is the standard workflow type and one is the express workflow type. So what is the difference between those two? It always depends on the use case which you are trying to implement, obviously. So standard workflows are good, for example for it automation report generation, order processing, or payment and billing processing, just to name a few. And express workflows are more like for event driven microservice orchestrations or APIs that need fast response. There are also different limitations that come with these different workflow types. So for example, the standard workflow can run much longer, while the exprs function can process much more events and functions at the same time. So yeah, it always depends on the use case what youre trying to achieve. But now let's look at some best practices and workflow patterns which you can use to integrate serverless workflows and step functions. So one easy one is of course error handling. So you can catch errors which can be thrown for example by a lambda function or by another integration. So you can then transition to a different stage if you catch an error which is unexpected for example. Then you can also retry with errors. So if one task fails, youll can just try it again to reduce the complex of your workflows. Youll can also have child workflows. So from one task you can call another step function and wait for it to finish and then continue in your main step function. That makes it easy to build more complex step functions. Let's look at some more use cases where you could use step functions well. You could use it for data processing, it security and automation, machine learning, or microservice orchestration. You as a developer might also be questioning what tools you can use to work with step functions. So let's look at developer tooling. So as I mentioned, there's workflow studio which is like a low code visual workflow designer for drag and drop, and it helps you to generate these RSL. It has three different sections, one where you can browse through the states, then you have the canvas and on the right hand in the inspector panel you can configure the different state. However, as a developer you might be using visual studio code, so there's also a plugin for it where you can build the state machines locally, visualize them, and also execute them on your local machine. Of course, you can also use your favorite infrastructure as code to build these workflows, like the serverless application model or CDK. As I mentioned, it's possible to also run these workflow locally using step functions local, which you can just download and install and run it on your local development environment and mock the SDK service integrations let's talk about a few best practices when using step functions. As I mentioned, working records from a use case, choose the right workflow type, then use timeouts to avoid stuck functions. Finally, when you have larger payloads, store the file in s three and then just pass the file handle. Also avoid reaching history quota and handle lambda service exceptions. For example, when your lambda function gets throttled, try to handle that exception also in your workflow. In closing, I have some more useful resources for you where you can play around with step functions. And please reach out to me. Feel free to connect with me on LinkedIn and I'm happy also to answer any of your questions. So thank you very much for listening and looking forward to seeing.