Conf42 Site Reliability Engineering 2022 - Online

Build the next evolution of composable architecture using GraphQL

Video size:

Abstract

With an ever-growing number of APIs, databases, and applications to interact with AND an ever-growing number of consumers that need to access data from them, companies are realizing they need a way to streamline consumption from all these new data sources. But where do we go from here?

This is where technologies like GraphQL truly shine - they are the evolution of consumption of a composable architecture. Using GraphQL, you can easily unify a massive number of APIs into a single endpoint, which is governed by a unified schema. This unified schema allows consumers to request data from ANY field that exists within ANY of the API’s associated with it. This means actions that would have normally taken 5, 10, or even 20 different queries can now be done in just 1!

In this session we are going to see how developers can automate the API unification process, how you can leverage a GraphQL endpoint to federate your REST APIs, and a quick peek behind the curtain to show how MuleSoft’s GraphQL solution, is able to provide you with the performance you expect.

At MuleSoft, we’re helping customers go digital faster by turning every asset in the organization into reusable building blocks. These building blocks are the foundation of composability. This concept of composability is seen first-hand through MuleSoft’s API-led architecture. In fact, the API-led or “API-first” approach works so well that it has pretty much become the industry standard. However, innovation in how development teams use and interact with the application network is far from over.

Summary

  • Priya Singh is a senior senior solutions engineer at Mulesoft. He talks about how you and your organizations can build the evolution evolution of composable architecture. In the future he would like to be a professional chef and get his private flying license.
  • Every organization is either building APIs or consuming APIs in one form or another. Developers need to consume data from multiple APIs all at once. That is exactly why we are talking about GraphQL today. A graph of APIs which will unlock new efficiencies.
  • Endpoint data Graph is Mulesoft's implementation of the GraphQL technology. It is a no code SaaS solution with an easy to use GUI to federate your rest APIs into one graphql endpoint. Anypoint data graph is not a replacement of GraphQL.
  • In today's session, I'm going to be showing you three core concepts. The first part is automating the unification of multiple APIs from your backend into one graphQL endpoint. And lastly, I'll show you some but of the box performance that you can get by implementing a graphQl solution.
  • Sana is a mobile application developer at your favorite retail organization. She has been tasked to expose the order status and the expected delivery dates to your customers on their mobile application. Sana's organization has started using anypoint data graphQL using endpoint data graph. This will make Sana's life so much easier.
  • Sana combines data from the sales API and the order API. Address, order status and expected delivery date are now combined into one order object. Collaboration is enabled because you want this data type to be added to other schema queries.
  • GraphQl allows front end developers to easily integrate APIs into a graphql query. You can use that graphQl endpoint on your postman or arc or any other API testing tool that you have. Using metrics, you can even further optimize your queries.

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Hi everyone, thanks for joining us today. Today we are going to be talking about how you and your organizations can build the evolution evolution of composable architecture. Graphql Graphql but before that my name is Priya Singh and I am a senior senior solutions engineer Mulesoft. I live in Raleigh, North Carolina. I am a Dei advocate as well as sustainability champion. I am also a home gardener. I grow flowers and vegetables in my backyard container garden. I also call myself a world traveler. And lastly, talking about the future, I would like to be a professional chef and at some point get my private flying license. Enough about me, let's get into the session. So we are talking about composable architectures, which means we are talking about APIs. Every organization is either building APIs or consuming APIs in one form or another, which means that developers need to consume data from multiple APIs all at once. To do that in the past to get data from multiple sources in the past, we were writing custom code to be able to collect the data and write it into a presentable format. Every single step was a custom code project. It would start from scratch and in the end whatever the deliverable was was restricted to that particular project. But today we are not doing that. At least I hope most organizations are not doing that. Today we are talking about reusability. We are building APIs for one project and reusing them in different projects. We are using multiple APIs to get data from multiple backend systems and then writing maybe more APIs to collect them and present them in a required format. But still we are reusing one API at a time and to extract data from the multiple APIs. We are writing more APIs on top of it to massage and transform and orchestrate that data. So that's the present state. Looking at the future, we would like for these consumption of multiple APIs to happen all at once. We have passed these custom code. That's a non starter. We have moved into the world of rest APIs, which is a great start. But we can only reuse one API at a time and this translates to multiple requests and custom logic to extract the data we need for our projects. Our aspirational goal then is to reuse these multiple APIs at once. Like I said, get data across these APIs at one time with just one request and no additional custom work to get what we need or need to write more APIs to create what we need. And that is exactly why we are talking about GraphQL today. So what is GraphQL? It is in the simplest form a query language for APIs where you can query data from different sources in one single API. Call if you are familiar with SQL, it is very similar to writing a join or a query which fetches data from multiple tables only. In this case we are fetching data from multiple APIs and because it is a very low code, no code implementation, we are talking about clicks and not code. And that's the reason for its popularity. Another benefit with GraphQL implementations is that it solves the problem of over and under fetching. So for example, when you are collecting data from multiple APIs, you have to parse through every API one at a time and then extract the data, read the data taken, write an API to be able to filter what you need and taken present it to the front end. So that problem is also solved by GraphQL. You don't need to write that logic which will remove what you don't need and only pick what you need. So let's go one level deeper to explain what it means to query these APIs. A graph of APIs which will unlock new efficiencies. So if you look at the picture, this may look similar to your organization's implementation. You have ERP information, customer information in either your ERP or another backend, maybe a CRM or another system, you may have shipment information, maybe in a database. You may have your product information, maybe in SQL server. So data is coming from different sources and all of these data models or data objects are its own thing. Now of course, nobody is just using for product information. Nobody is just asking for shipment information. We are asked to combine the information and then present it, for example, order. So an order information will of course contain shipment id, the customer's information, the product details of what is part of the order, and all of that. So that is then another API which pulls data from the customer API, from the shipment API and then creates that order API that our developers might use. So if you looked at this carefully, we can see that these individual APIs often have relationships with one another. For example, in these picture, the order API is related to the customer API through a customer id. An invoice API is related to the product API through a product id. And that is interesting because these relationships, we can conceptually stitch them together and form a graph to build all the capabilities within these APIs in one shot. And that's graphql, the graphql of stitching these APIs together. Now this big graph will unlock many new possibilities. We can use this graph more efficiently to reuse what is already existing in our organizations. What does that mean? It means that tomorrow, if a project is asking to show orders, the order status and the expected delivery date, we will not write another API to get the data from the product API, from the customer API, from the shipment API, and the order API, and the invoice API. To give that view, we will actually write queries instead of APIs. That's what we are talking about. The graphql is actually the next evolution of consumption of a composable architecture. Using graphQl, you can easily unify a massive number of APIs into one single endpoint, which is governed by a unified schema. This unified schema is basically the definition, the combined definition of all the APIs that you are connecting together. In the slide here, you are seeing that we are connecting product, order, customer, shipment API into one graphql. Imagine how many data points can now be gathered at the click of buttons. This unified schema allows consumers to request data from any field, any field that exists within any of these APIs that are associated or connected into the GraphQL endpoint. This means actions that would normally take 510 1520 weeks will now be done in probably or possibly a day. So if you look again on the slide, the store API, the marketing item API, the customer API, all of that is part of one big graph. And just like any graphql, you're just going to write a query and pick what you need. So now I hope you understand GraphQl and what we are trying to solve with this. Now I will talk about anypoint data graph. Endpoint data Graph is Mulesoft's implementation of the GraphQL technology. In short, it is a no code SaaS solution with an easy to use GUI to federate your rest APIs into one graphql endpoint. You don't have to write multiple API URLs and endpoint to get the information you need. It will all become one endpoint, which is a GraphQL endpoint. It also offers very easy to use GUI and I'll be showing that to you as part of this session. And you can define very easily relationships between these multiple data sources, customer, shipment, product order, whatever you want. So anypoint data graph, I want to clarify, is not a replacement of GraphQL. Instead, it is meant to enhance what GraphQL is incorporated or how GraphQL is actually incorporated into the software development lifecycle by tightly integrating user experience into that. More specifically, data graph is meant to be an easy to use GUI for defining relationships between multiple data sources and multiple APIs. So with that said, with that explanation done and clear, let's see what we are going to see as part of the demo today. So in today's session, I'm going to be showing you three core concepts. The apifirst one is how you can automate the unification of multiple APIs into one graphQL endpoint. And then I'm going to show you how you can easily retrieve data through that GraphQL endpoint. And lastly, I'm going to show you some but of the box performance that you can get by implementing a graphQl solution. So let's get into it. The first part is automating the unification of multiple APIs from your backend into one graphQL endpoint. We will start off with that, it will be quick and easy and we'll show how it's done. To make this a little more interesting, I'm going to be following the story of Sana. So Sana is a mobile application developer and she works at your favorite retail organization. Could be Macy's, Lowe's, Walmart, Publix, Zara, whatever your favorite retailer is. She's a mobile application developer there. And as part of their digital initiative, Sana has been tasked to expose the order status and the expected delivery dates to your customers on their mobile application. So what she needs to do, as you're seeing on the screen, is this is the interface of the mobile application and within these customer's my order page she has to display the order status and the expected delivery date. So let's see how Sana would have done this in the past. So traditionally to accomplish this, Sana would have to first identify what APIs she needs to access. So she needs the expected delivery date, which means probably needs a shipment API and she needs the status, which means probably the order API. So she will have to identify what APIs she needs access to. And hopefully those APIs are already implemented by the IT teams and the development teams. Once she has identified, she needs to request access to those APIs and make sure that she can get the data out using those APIs and the APIs to be available to her for consumption. And lastly, once she has gotten access to those backend APIs, she will write new code or new logic to filter out only the information that she needs. For example, order API will not just have order status, it will have order number, it may have customer details, it will have product details, it will have a lot of other extra information that is irrelevant to Sana. So she will have to write the logic to just pick out order status from it. Similarly with shipment API that will have delivery information, the partner or the carrier, and then the carrier's details and the order details and could be many other information that is there. But she will have to write the logic to just pick out the expected delivery date for her order or for the customer's order. So think about all this. I mean, I come from a development background and I know we are talking months. We are talking months to identify the right APIs, to get access to those APIs, to make sure I can get data using those APIs. They are still working and consumable and to write the logic on top of it to exactly build the view that I need on my mobile application. Yeah, we are talking months. But the good news is Sana's organization or your favorite retailers have started using anypoint data graphQL using endpoint data graph. What these backend teams have done is they have already created that unified schema for the combination of order API, sales API, shipment API and now they are sitting as one graphQl endpoint. So all the order information, sales and shipment information is already collected into one graph. This will make Sana's life so much easier. She is not just saving months and weeks of effort, she is saving so much work that she needs to do from scratch. So this is great. And this is where my product will come into the picture. I'm going to show you how quickly the backend teams can easily combine those APIs for Sana and present her with these unified view and how she can then easily query to get the information that she needs for her project on the mobile application. So we are looking at unifying order, sales and shipment API and then create that unified schema. So let's get into the demo and if I look on the other side of the camera, I'm just going to my browser so that you can see the right window. So this is any endpoint data graphql once you land on this page, this is the view that is presented because Sana's organization has started using endpoint data graph. So here in the overview you can see that it is already deployed in production and the endpoint is already here. This endpoint is that one URL using which you can access all those APIs, the shipment API, the order API, the sales API and whatever other APIs is linked into this one graphQL endpoint. You have one URL to access all of that. So now you can see the endpoint here. You can actually use these endpoint in your favorite tool, like any tool that you use for testing your URLs using a curl command or any other method, she can also go and look at the unified schema that was created. And the good news is that the status is up to date. This means this is relevant and it is already in production. So this is how Sana will log in here and see that the data graph is already created. Now let's see what that unified schema looks like. So for Sana, she wants to have the order status and the expected delivery date. So she will start querying what data types and fields are part of the unified schema. So if I click on these unified schema, I see that I have the query operation available, which means that I can get information, but of these back end APIs using the endpoint that was given, if I look here, these are the methods that are available to me, which means these are the operations that I can perform on the endpoint. So I can get customers address by using the iD. I can get customer details by id, order information, shipments, address, customer and order. So these are the main high level operations that are provided to me to get data, but using the GraphQL endpoint. So these are the operations available to me and these are the data types. So if you see here, there is a level one type and there is nested type. Level one types are basically the high level data objects that you can directly get from the query operation. And nested types are the data fields that are part of the level one types. So it is kind of like one level below. So for example, if I wanted the delivery address of where the product is being shipped to, I cannot just query address by id and get the delivery address. The delivery address will be part of the address object and by queries address by id I can get the delivery address. Similarly, if I click on order, order will have many fields below and it may have some nested types. So these nested types you can get the data using the level one types. Let's go to order. So she needs order information. So she's going to inspect whether the fields that she needs are part of the order data type or not. So if I click on order, I see that it is a merged type, a merged type, like the name says, it's very self explanatory. It's a merged object or a merged data type combining the data from two APIs, two or more APIs. It doesn't have to be two. So in this case we are creating an order data object by combining the information from the sales API and the order API. Let's see what that means. So if I scroll below, these are all the fields that are part of the order object in the unified schema. So if you see here the order id, customer, email, delivery, all of these cannot be part of the sales API or the order API. So some fields here are coming from sales API, some fields here are coming from the order API, but we have created a high level order object to get that all data into one object. So along with all the fields, she also sees that she can get the orders by Id or you can also get all the orders of that particular customer. So these are the methods available in the order data type and these are the fields. So now she needs status and expected delivery. So she scrolls below and she sees that order status is here in the part of order data object. And delivery information is also available to her as part of this order object. So let's make sure that the delivery object has the shipment details. So shipment are part of it. Let's make sure that she has expected delivery date. So there you go. Address, order status and expected delivery date are now combined into one order object exactly as she needs. So let's recap all the things we saw. So we went to unified schema and we saw that. Let me go back. Yeah, so shipment type and then these were all the data types that were available to Sana. She looked at the order type and made sure that the two fields she needs are available to her. Now she's just curious, where are these two fields coming from? So she can click on list of APIs that are added and it will show her all the APIs that are part of these unified schema. So now she knows that shipment address, sales, customer and order, they are all part of the unified schema. She can retrieve not just the order status or the expected delivery date, but she can extract means more data objects using that same URL, the same endpoint. She doesn't have to write different methods or different queries. So let's investigate a little bit about the order API. So if she clicks on the order API, it will show all the data types that are part of the order query API. She can query by order by id, and here are all the part, again, the fields that are available to her. So we saw the order merged object. Now let's see what else is happening with the order object here. So this is the order. We discussed that it's a merge type, but we see that it's also a linked type. Let me show you what that means. So collaboration is enabled because you want this data type to be added to other schema queries. That's why we have enabled collaboration. And then these are all the fields we already saw, these are the fields that are part of the order merged object. And then here is the link that is created, the link to another type. So if I click on here, it will show me that this order object is actually not just part of the order API or the sales API, but it is also part of the customer API and customer Id is the foreign key. So if you understand SQL, this is basically connecting your one API to the other using a common parameter. Like we discussed earlier, an order API and a product API can be connected with an order id. Similarly, the order object is connected to the order API and the customer API using the customer id. So that's what we call the linked types. If you again scroll below, it will show you how it is merged into one and how the overall order object looks like. So she's sure that she is getting all the information that she needs, she's happy with it and she's satisfied that she should be able to write a query and get the data without having to write additional logic in her mobile application. So let's go back to the slide and recap what we just saw. So the takeaway from this part of what I showed you about endpoint data graph is how easily backend APIs can be added to one bigger data graph or one bigger unified schema or a graphql of those APIs combined together and how you can do it with clicks and not code. I also showed you how easily you can decouple all the back end access issues and these security and all of that from the front end developers so that they can save a lot of time and easily get the information that is needed for their projects. In this case, we are completely decoupling the backend data source and the data access issues from Sana, who is the mobile front end developer. And ideally she shouldn't need to be worrying about those backend things. We will, of course. So one thing you may wonder about the security is how we have decoupled the access, but how do we make sure that it is secure? So the graphql endpoint that I showed you, you can actually manage that endpoint like any other API. You can put a lot of security and governance on top of those APIs so that it is only accessible to the right teams and to the right people. So that was the first part. Moving on, let's see how Sana can easily retrieve data using that graphql endpoint. So we have seen how backend teams can easily integrate APIs into a graphql. Now let's look at how front end developers like Sana can use this. So for this part of the demo, I'm going to show you how Sana will log into data GraphQL and build that query and get the data that she needs. So let's get into it again, I'm moving my face because I'm looking to the browser, so don't think that I'm going anywhere. So again, let's go back to data graph. So this is where we are going to see how Sana will run the query and get the data. So for that I have to go back to my unified schema. And at the top, if you see there is on the right a run operation. So I will run that operation. Basically run operation in this case means running that query which will pull the data that Sana needs. So in the interest of time, I've already saved the query here, but you can use that graphQl endpoint on your postman or arc or any other API testing tool that you have and it will work the same way. All you have to do is put that URL in the URL bar and create a query like this. This is a very simple JSON query that you can run and this should give you the output that you need. So for example, I have saved this query and it gives me all the fields that I'm asking for. So Sana needs order status and these delivery date so she doesn't need tax amount. So let me remove that and let me show you another great feature about GraphQl. So right now I don't know what is the field name or what other fields are available for me to access. Now I'll have to pause. Just think about it. So I have connected order API, shipment API sales API, customer API. There were like five, six I showed you, right? They are all connected into one endpoint. So now I can just simply type a and it will show me all the APIs that are available for. I mean I can type any Alphabet if you think I need delivery info. So I just write road delivery and it will autocomplete the right field name for me. And I don't need to know which API is this field coming from because now all the fields that are part of those APIs are combined as one. So I can click delivery info and it's giving me the red which means that it's an object. So I can start writing delivery address. Actually instead of delivery address, let's do tracking, maybe shipment expected delivery date. There you go. So now if I run this query, ooh, I think I'm, I think I'll have to avoid having an array. So let's try. Would avoid having can array because I've ran queries too many times. So let's see what else I can use. Let's go back to tax amount. So this is how easily Sana can write a query and she can add as many or has less fields she needs. She doesn't need to write more APIs or custom logic in her mobile application to see the fields that she needs. She can just abstract all of that information, unnecessary information out of the query and just write the fields that she needs. So this is how easy it has. Now I'll show one more thing. So for example, let's take the example, let's say Sana is working for a very business critical application which is sending order consumption to the customers. As soon as they place the order, they should immediately get an order confirmation. And we all expect these days like very quick responses to our retail experience, right? So in that case, I am sure the application that Sana would be writing will be very closely monitored by the business. So she has to make sure that these query she's writing is responding very quickly within the SLA that the customers expects. So to make sure that it is working the way that we expect it to, she can go here and do a trace query. So now if I run this again, there you go. Not just it's giving you the response time or the throughput time of not just the API call, but every field. It is showing me how much time every field is taking to respond back. Why? Because every field might be coming from a different API. So now you see that the second record is taking a lot of time or tax amount is taking the most time. So you can then find which API tax is coming from and you can conclude that I need to optimize that API. So using simple like, there are so many metrics that are available here that you can quickly troubleshoot if you are out of SLA or you want to optimize your queries even further, you can do that very easily using a lot of metrics available here. So let's go back and recap what we just saw. So this is what the takeaways were. We saw that how Sana and other developers, front end developers, can streamline data access from a single graphQL endpoint. They don't have to use multiple URLs or APIs and they are completely abstracted from the backend APIs. These can easily update data. They can also improve the performance and optimize their API calls or their queries to make sure that they are within slas. So a lot of that time that would have been spent in writing custom code and finding out the right thing that we need to grab and how to grab it, she can spend time on innovating. So now let's look at the mobile app, what SAna has used for the query. These it is. So if you return the right, you have the shipment tracking information, you have the order details, the status and everything with clicks of buttons. So now, instead of focusing on building logic to parse the data across APIs, developers like Sana can focus on actual innovation. They can discover and reuse the same single endpoint to surface the information on the web or in the mobile apps, saving them hours and hours of work. So final takeaways we saw the automate unification of APIs into graphQl and how backend teams can quickly implement a graphQl solution, how front end teams can leverage the unified endpoint to access the backend systems, and we also saw how customers can easily receive the performance they expect with out of the box functionality. I hope you enjoyed the session and if you would like to learn more, I have but some sources here. If you scan the QR codes, they will go to a URL and you can find the right sources there. Data graphql is free to try. So you can scan the code and it will take you to I think a trial account registration. So you can do that. If you want to get into some details about datagraph, there is also a tutorial I have linked here that you can use to maybe build your first project. Just get a little hands on on how it works and then if you want to learn more about GraphQL and its documentation, the third QR code will taken you to the GraphQl documentation. And the last one is a little bit of product overview, more about endpoint, data graph and graphql and what it means and why businesses should know about this and make the best of it. I mean these days every popular company that you know of, Pinterest, Twitter, all of them are using GraphQl in one way or another. It can find its use in many, many ways. So I hope you enjoyed the session and I hope you learned something interesting and something new. Looking forward to interacting with you. Thank you and have a really great day.
...

Priya Singh

Senior Solutions Engineer @ Mulesoft

Priya Singh's LinkedIn account



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways