Hello, everyone. My name is Adrian Machado. I'm a staff engineer at Zupelo API management, and I wanted to talk to you about why your API needs to operate at the edge or the worldwide edge, as it's sometimes called. In this talk, I'm going to go over quite a few things with regard to the edge. So if you're not familiar with the edge or serverless, don't you worry, I'm going to cover that for you. I'm also going to cover some considerations and trade offs that you should consider, before you adopt the Edge. It's not just like a silver bullet that's going to improve the performance of your APIs or microservices. I'm also going to talk about why APIs, specifically, are a good Application of the edge, why they're well suited to be deployed to the edge or at least parts of APIs. And then finally, I'm going to end with a quick demo of an edge API that I built, and it's going to help showcase some of the trade offs that I mentioned earlier in the talk, as well as how you can debug performance issues when building an edge API. So let's jump into it. So who am I? I'm Adrian. Like I mentioned earlier, I'm a staff engineer at Zuplo. I love writing about all things API related, and I also create a lot of open source API tooling. One thing I created recently is called RateMyOpenAPI, which is an open source API governance and scoring tool. I'm also a big fan of shoehorns, just decided to throw that in there. If you don't currently use a shoehorn to put on your shoes, you should. It's definitely a lifestyle enhancer. I like to qualify. The people attending my talks, just so you don't waste your time, 20, 30 minutes on a talk that you don't find useful. So I think you'll enjoy this presentation if you're one of the following buckets. If you're like an API engineer or developer that's looking to modernize and legacy API stack using the latest and greatest, I think this talk is good for you. If you're an executive, like a CTO that oversees API development, you're learning, techniques and architectures to help improve API developer experience at your company. Whether that's internal APIs or external APIs, but especially public facing external APIs, this is a good fit for you. If you're just looking to build an API for the first time, and you don't know the best practices and tools that you should be using, this is also a good talk to you, and this is not a very introductory level, but it's a good high level concept around the edge, so it's not gonna get too in the weeds. So let's start with what exactly the edge is. So when you typically deploy a microservice or an API, your application or API is deployed to a single region or a data center that's EU West one, or maybe that's NA East two. Wherever it's deployed, it's in some geographic, region and. Unfortunately, we operate in a world where our users can be anywhere, so all of these requests from around the world need to be routed through all these, undersea cables, and eventually they get to the your data center, and by the time the response gets back, it's gonna have pretty high latency, sometimes in the order of seconds. If a call is coming from, let's say Japan and hitting your data center in Munich. So there are some ways of getting around this that has have been explored in the kind of application development space slash front end development space, CDNs, which stand for content delivery networks, were created to host the static content that is closer to users. So think like images, JavaScript, fonts, all of those will be hosted in some local data center in Japan, and when you load a web page, all of those will be able to load faster, and. those are typically chosen one because there's static content, so they can be essentially cashed and two because they're typically large files that would take a long time to deliver over a network. But a problem arose, this is all well and good for websites over for dynamic content like code. Your compute, essentially, is going to be slow. the actual processing of the website, the rendering of the webpage, that can't all be cached because it's essentially rerun every single time. a bunch of, CDN creators, like CloudFlare, Vercel, who works a lot in the application space, they all got together and they formed like a working group called WinterCG and they came up with this idea around deploying Code to the edge. When I talk about code, I'm primarily talking about function as a service. So if you're familiar with AWS Lambda, for example, or Google Cloud Run, then you'll know what function as a service is. Essentially, it's just like an isolated function that is spun up. So realistically, it'll execute some code and then It'll be essentially put to sleep when it's done and another request will come in and wake it up again. So these are, this code is actually distributed at the pop layer. So it's essentially at the same level that your CDN operates in. And what's nice about this is it provides like a lightweight runtime that typically has a zero millisecond cold start. When I was talking earlier about latency, one characteristic we talked about was, the round trip time to even get to the data center, but if you don't have an active instance of your application or like essentially a warm VPC or a machine, essentially, or server that is, actively taking requests, if a new one needs to be like spun up to handle some requests, that's going to take a lot of time, especially with how heavy, Run times like no GS can be. So what's nice about edge is that typically this run time is super lightweight and has a zero millisecond cold start. So not only are you close to your user, your application code will actually wake up and execute faster, which is good for instant responses like APIs and web pages. There are some tradeoffs, though, with most Edge runtimes. I'm going to go into some a little bit later, but your bundle size, the languages you can use, and even the libraries you can use are limited. And some examples of Edge compute platforms include Clever Workers, as well as the Vercel Edge for web development. So now let's go into those tradeoffs that I mentioned because the edge might sound good, it sounds great to be able to run your code essentially as close to your customers as possible, but it's not a silver bullet. So let's start with the edge runtime. So the edge runtime primarily the one that I'm familiar with is the one developed by cloud flare. It's Currently limited to JavaScript only right now with some support for WebAssembly. So if you're, someone developing an API or application in Python or in Rust, for example, you're out of luck unless you can compile that down to WebAssembly. Otherwise, if you're doing You know, your basic work in like scripting application, but not like Node. js, you should be fine to run your application at the edge. there is some limited support for Node. js libraries, but most of these are done by hand by the Winter CG group, so you can't just. Import a Node. js library and expect it to work out of the box. A lot of times if it, relies on like TCP or other networking APIs that Node. js exposes, it probably won't work. And there's also limits on the maximum bundle size. I think It's something like 100 megabytes or 10 megabytes can remember off the top of my head. It depends on the platform you're executing on and it always changes over time. But yeah, you can't have a massive application running on this function as a service, which is similar to existing fast applications like Lambda. I will say, though, that things are getting a little bit better. There are many libraries out there that now have support, or they have, essentially, an edge version of their library that doesn't rely on Node. js primitives. And any library that's built to run in the browser for JavaScript can typically be executed at the edge as well. Because the edge is essentially a version of the V8 runtime that's used in your browser, which is why it's so fast and lightweight. So aside from complications with the runtime, a big factor on why or whether or not you should use the edge is where your data is located. So your compute may be globally distributed, which is super nice, but your database and or data store, wherever you're storing your data is often not globally distributed. And this results in actually. pretty poor latency characteristic when you're doing something like a crud operation because you have to account for not just the time to access your function, but the round trip time from like your edge location to your database and then back from your database to your edge function, which can often lead to worse response times than if you just Co located your function and your database in a single data center. It depends where the call is coming from, but this is often the case due to routing. You can also end up exhausting connections to your database as well. You can imagine, if you have users all around the world, they're all invoking all these different edge functions. If you're doing something like connection pooling, you can probably easily exceed the number of connections that your database allows. And. you might just end up stuck or locked out. And again, this leads to worse latency. I'd like to mention that there are answers to all of these kind of issues, and I'm going to start covering some of them right now. So this is an image that indicates, demonstrates what I'm talking about. You have this central function in the middle here that's really nice. But then, you have your functions over here and you're trying to return data to users. And it's just a long trip from your function to your server over here and then back through your database. the easiest to adopt way to mitigate this problem is to introduce a database proxy database. A proxy is essentially a kind of layer that will be like a gateway, to your database, where the functions as a service or edge functions will connect to it instead of database directly. And then this database proxy will field all of these connections and maintain its own pool of connections to connect to the database. So it won't exhaust the connections to your database. And you end up with crashing on your Postgres or anything like that. So there are solution managed solutions out there like Prisma Accelerate. I think AWS has something called like RDS proxy as well. And, this is like an easy solution is you don't have to change anything about like your database configuration. From my understanding, you can just simply start connecting to a proxy instead, and it'll manage all the connections from your edge functions automatically. yeah, so like I mentioned, this is good if you're like transitioning from an existing database, and you can also introduce caching at the database proxy layer as well. Rather than, using one of the connection pools, if you if your query is something that is cash, then it could return it directly. And that's the best case possible is if your data is stored somewhere close to your function. There is another solution out there, which is essentially having a global data store. So that would be like distributing your database across multiple regions, which is very common in high scale systems, I'd say, in order to minimize latency, whether you're using the edge or not. So you can use a specialized type of database like Cassandra or CockroachDB, which can be placed closer to your edge function, not exactly in the same place. Layer. you can imagine you have a database for Japan and then you have edge locations in Tokyo and Osaka that will connect to that. And it's going to be a lot less of a round trip time to get there. The trade off here is going to be primarily with. consistency, depending on how you do your writes. So if you're doing like asynchronous writes, and this is just like a read replica that you placed in Japan, you can potentially have stale data because it's going to take time for the writes that happened in, let's say, Munich, to propagate to your read replica in Japan. if you do them asynchronously. Or, you can have issues with latency. If you're doing synchronous, right? So if you hold on to your database and your replica and lock it while you're performing rights, then you can have slow reads because of that database replica might be locked while you're serving API calls. So there are definitely tradeoffs here, but typically this can result in better performance and kind of solves the issue. just like a basic image of you have your main data store somewhere in North America distributed and Europe and Africa and all over the world. There's a third solution, which is a little bit less common, I'd say, but it's becoming more popular. And that's the idea that you know, your data should live where it needs to live, where it would provide the best user experience. So in some cases, you can have data that is hosted or cached at the edge itself. And there are different kind of use cases here. So say you want to do some sort of like blob storage, some sort of like cash, right? you can use a solution like Cloudflare's R2, which is essentially an edge distributed, edge, blob store. if you want to do Database transactions and want to have something like a per user database or per instance database. Maybe you want to have a database that's specific to like a region for GDPR compliance or something like that. Then you can use D1 from Cloudflare or Upstash to have an edge distributed SQLite database that lives very close to users and is locked to a certain region. Lastly, if you want to do something like, key value store, Redis and Vercel both provide solutions for their for like key value store, which is, good for like configurations, for example, that you don't want to travel to a central location for. But again, there are going to be that kind of trade off of consistency. So this is ideal for configurations or data that doesn't get updated very often or doesn't need to be like replicated, So this is a happy path where each kind of instance has its own database over here and everyone lives happily ever after. So that's a basic idea of the tradeoffs of deploying an application to the edge. But why should you deploy your API to the edge or parts of your API to the edge? when it comes to APIs these days, especially public APIs, I think performance matters a lot. Your public API users can be anywhere. And, this is by nature of you actually have no control over what your customers do. Once give them like an API key, they can deploy, their application, which uses your API anywhere, whether your API customers in Japan or China or in Africa, you don't know where their customer is going to be, but you're going to have to serve all that traffic all the same, right? And if you don't have good performance characteristics in a certain region, that can be like lost customers for your company, for example. and another thing that is nice about APIs, I'd say, is how compute heavy they can often be. So I'd say like a lot of work done by APIs is often just filtering out calls. So that's, and I'll get to some examples later, but like a lot of time spent within an API, besides like the actual like CRUD and like data portion of it, is just like compute. If you're doing a lot of compute based work, that's good to actually move that portion of your A P I code to the edge. So to give you like an example of the tail of two calls over here on the left, we have an example. If you're doing like a single region where you know your data centers history in Dallas, Texas and your A P I calls coming in from Japan, you have some customer over there essentially just to do something like Yeah. Authentication even if you're denying the request. Let's say they forgot their API key. You have to travel all the way from Japan over to Texas over here to do your off handshake and then you travel all the way back just to say 401. It's not a very good experience. Versus over here when you deploy to earth or that's commonly known as like how to deploy to the edges to point earth because you deploy globally typically, you essentially can have this edge based proxy in front that can do stuff like authentication, authorization, a lot of again, the compute based stuff that can result in like quick returns of responses in case of invalid data or invalid, like tokens are sent over. So examples of work that can be done at the edge that's compute heavy, authentication and authorization unless you, if you're, especially if you're in housing this through something like API keys, or, anything basically other than using like a, like an identity provider, if you're doing it, Yourself or you're writing the code yourself doing it at the edge is a lot faster. You can respond with that for a wine or 403 so much faster compared to having to travel all the way across continents. If you're doing logging and monitoring within your API, doing that within the edge is. Again, going to be faster. it's just code being executed. Request validation again is something that's going to be a lot faster. You can use libraries like AJV, for example, or Zod at the edge, and you can do schema validation of like request body and filter out like that request or SQL injections or any kind of like bad stuff that like a WAF would catch web application firewall. so Bot filtering or bad actor filtering or even filtering out agents these days. you can do that all at the edge as well, because that's all just like computations being run. More interesting to be done at the edge is rate limiting, for example. So rate limiting at the edge will actually have a win effect. The win, I'd say, is that since you're doing it, since you're filtering out at the edge, it doesn't make it to your central service at all. so wherever you're, Your actual like API and microservices are hosted and that reduces the number of requests going to them, which frees them up for with more resources to actual handle legit requests. So you might actually end up with like better latency by introducing rate limiting for your customers. And you also filter out, like DDoS attacks and stuff like that. The last one I want to mention that is really good to be done at the edge is caching. So if your API is amenable to caching, so let's say it's like a basic like trud, that first call may take like a while depending on where your data store is, but subsequent calls to that Same API will have like very good performance because essentially it's stored at the CDN layer. It's cash. your user in Japan doesn't need to, the request doesn't need to leave Osaka to get like data. So like refreshing a webpage, will be very quick or navigating back to a webpage that you previously loaded. So I'd say like my ideal setup and the one that I advocate for when it comes to running APIs at the edge is as follows, essentially. You run all of those functionality that I mentioned earlier at the edge, and in that case, that should be done probably at the API gateway layer. So you should have, an API gateway that can run at the edge, and that's what I work on at Zuplo, actually, is developing an API gateway that can run at the edge, whereas your microservices that actually, serve your API traffic are hosted somewhere close to where your data store is, and ideally replicated across multiple regions. So you would have, a read replica of your database, as well as a kind of like function as a service or Lambda of your actual microservice hosted, one in the US, one in Japan, one in Germany, and you would have these, your gateway distributed to specific cities like Munich and in, I don't know, London and London. Maybe the west coast of the U. S. And the east coast of the U. S. you can do a lot of that filtering at the edge, but the computer is actually done within some central server close to your data. So you can minimize latency on this kind of gives you a lot of freedom, too. So you're not limited by having your microservices be bound by like the restrictions of the edge runtime. You can write your code in like Django or Python or whatever you want to do. That's fine. It's just that the gateway layer would be written in like JavaScript, essentially, or wasm. And, Yeah, this is probably like the ideal scenario. I wouldn't say many companies run this way yet, it's becoming, I'd say, increasingly popular as APIs become like a product more than just like an infrastructure piece. And, a company like Scribe, for example, that sells their API and runs at a huge scale probably wants to, pursue a solution similar to this. as I promised at the beginning. early in the talk, I'm going to give you a quick demo of an edge API that I built when I talk about an edge API, essentially, I tried to have as much as the code and the data live as close to me as possible where my local data center is. The application I decided to build here is an ATM locator, so if you ever use like a banking app, trying to find like an ATM nearby, and you didn't use Google Maps for some reason, I don't know why you wouldn't, but, this is in every banking app I'm aware of, essentially you find all of the local, ATMs of Capital One, let's say, nearby, and, what's nice about this and like why I decided to build this at the edge, a few things. One, ATM locations, essentially, essentially, They can be like replicated in store at the edge because, one like location of ATMs are specific to a specific like region, like you're not going to look at, ATMs in London when you live in New York, that just wouldn't make sense. there's no reason why like your data needs to be like. totally replicated, for every single user, and it's also really written to like it takes a long time. I assumed to install an ATM. So the API or like the data store for writing to a specific regions ATM list probably doesn't happen very often unless you have something like I don't know, like downtime monitoring or something like that of your ATMs. But again, I'd say like the number of right query is going to this is going to be very low. I also think that the performance of this API is going to matter a lot. Someone's probably, looking around, probably needs cash quickly for settling up like at the bar or they forgot their credit card at home, but they have like their debit card for some reason, and they just want to find an ATM quickly. they want to open up the app, be able to find the closest ATM super fast, navigate there. In your instant time, essentially, and another good thing about this, I'd say, carrying off like the previous point is that you can cash these results super easily since again, it's like very local and rarely updated. So you can have a long lived cash for best performance. In terms of the data set I'm going to use to build this API, I'm going to use Capital One's Nessie Hackathon API, which is essentially just a virtual bank. it's like a subset of their data, of their banking data, including stuff like ATM locations. And, one thing I'll mention is that their API performance is actually already really good. The reason why I choose, chose Capital One actually is because I know they're a company that's already adopted serverless and edge functions for their APIs at scale. And I'm just going to quickly show you what that looks like. Essentially, if you fire a request against, their API, they can return, something like 10 results. They paginate it, but they return, 10 results in, 100 milliseconds, which is pretty fast, given that, I don't know exactly what this data is. I'm just going to quickly Exit out of my slides over here. I'm going to pop over to the Capital One website. I'm going to call their API for ATMs over here, and all you do is you can put like a latitude and longitude and then a radius in miles, and then you just click try it out. And on the right over here, it's going to be a little bit hard for you to see, but let me zoom in a little bit and pop this open and you can see the response we got back was within 171 milliseconds. And probably if you click it a few more times, it's even faster. There we go. Within 89 over there. So that's a pretty fast API. I'd say, this is like sub 100 milliseconds average case, except the first one. Probably that was the cold start. That was talking about earlier, there is probably a server that's. like a sleep, up there, or maybe the data store was just like further away. And there, this is with caching over here. So overall for, a hackathon API that probably uses some production infrastructure, this is pretty good. Okay. So now I'm going to try and build my own version of this API. I scrape the data set off of capital one to get all of the list of ATMs, and I'm going to build my own. API locator, or sorry, ATM locator at the edge. So what's that going to consist of? that's going to consist of the following. We're going to have a simple Brazil website that's hosted by, it's in Next. js, hosted by Brazil, wherever it's going to end up hosted, probably somewhere in the U. S. And that's going to call an API. that I've set up and that API consists of two parts. It's going to consist of an edge cache, which is going to cache the results for me at the edge. But before it does that and on the initial call, it's going to invoke essentially a simple edge function that is connected to an API route. This is done through Zooplo. And then I need some data to host the, ATMs. So I have a D1 database over here. That's the one I mentioned earlier that Cloudflare produces. That essentially is like a just highly replicatable, distributed SQLite fork. that's basically the gist of it. you can read over this in case any of that didn't make sense. But, in the best case, I'd say is that, Purcell just, Places the instance of the website somewhere close by. Zuplo finds an edge location that's also close by, to perform the function. And then, ideally, D1 finds a place to put the SQLite database, also close by, somewhere close by to those functions. ideally, all three of these are close to each other, and we have very low latency. let's jump into it. I have the sample over here. This is my edge API locator. I'm going to refresh it to prove to you that it's real. I have a demo location here in McLean, Virginia, USA. That's just where the dataset is. And I'm going to search for ATMs within five miles. Just the same thing is like the Nessie example that I showed you earlier, and there you go. We have some examples over here. These are some of the closest ATMs is able to find. I guess there aren't that many in McLean, Virginia there. There are a few times over in Arlington, but one thing you'll notice that actually took quite a bit of time. If you ever over here, I think they'll tell you it took about, let's see. That took about 1. 34 seconds, to get the results. That's actually a lot slower than what Capital One took. If you remember, Capital One took, essentially 90 percent less time, about 134, milliseconds to get results. why was that so slow? Let's pop over back to the slides, and we can start debugging things. So in any, function as a service, there's going to be some sort of slowness. so we should try and debug it and try to rule out what could be going on. the initial call, like I showed you, took between, 1, 000 and 2, 000 milliseconds, which is obviously too slow for any, user, especially when I just said we were prioritizing performance. so here are some reasons of what might be happening. So the first one is that, D one itself could be slow at executing queries like this. The sequel query is essentially just like a between queries where I have all these like latitude and longitudes set up and have to find like a all the results between these two sets of latitudes and longitudes. And, we can have a lot of data. I think there's like several hundred ATMs here. that shouldn't be too much for, a typical database, but I don't know how fast D1 is. but, yeah, there are different ways of debugging database slowness, for example. And, probably introducing an index based on longitude and latitude would be the way to resolve that, if the query is what's taking a long time. their most likely suspect, I'd say, is that D1 was simply placed on an instance that's too far away. and this is typically because Cloudflare will only assign you a, high value, high performance database instance if you're sending, high volumes of traffic, essentially. So this is actually, this is not very good for hobbyists that I say, and, if I knew D1 did this before I started building this demo, I probably wouldn't have used, D1, but here we are. the best way to get around this. is probably the cache results at the edge. So you have to make that round trip as few times as possible. The last thing to rule out, I'd say, is API gateway latency. that would be like, latency introduced by the function as a service from, Zuplo and compute that's being done. And even just the routing of, that API endpoint. and probably, if, if that's the issue over there, you should probably move to another platform or move off the edge. there's clearly something wrong. Maybe you're, maybe you should just use a different language than JavaScript. I don't know what's going on there. Let's, pop over here. I've performed these kind of tests already, just in case that the demo didn't work and didn't want to didn't want to bore you. But essentially what I did is I made a call to Cloudflare's API directly. This API is essentially allows you to access your database for like reads and writes and. Whatever types of queries you want to perform. this is my SQL query over here. essentially I'm just selecting star from my ATM list where latitude is between two values and longitude is between two values. And I'm limiting down to ten results so I can match what was done by capital one. And here are my parameters, that's my latitudes and longitudes. don't worry about how I computed this, there's a, there's something called like the Habersine, algorithm to determine latitudes and longitudes between two points that are within a certain radius. just don't worry about it. But, what you should worry about, I'd say, is these numbers in greens. Green over here. You can see the result size is not like too large, just a few kilobytes, right? The things to look at. One, that query to Cloudflare itself, so that's excluding the API gateway or whatever latency it brings. That took alone 830 milliseconds, which is most of the time of our call. I've run this a few times. It can take up to 1. 2, 1. 3 seconds, for example. So this is like the bulk of our duration over here. It's just the network request. And if we were to look at What the component of that network request is it purely like network latency or SQL code execution time? What's nice about Cloudflare D1 is that they actually send you back this property called Duration where they tell you exactly how long the query took to execute and in this case it took 0. 24 and that's actually I think in second. or maybe that's in, sorry. Yeah, that's in milliseconds. So it took 0.24 milliseconds to execute that query. So pretty much all of this was from network agency and whatever is going on in Cloudflare's, API, whatever they have, going on like checks and authorization or authentication, things like that. Their API itself and accessing over like a network, the internet, is taking a long time. So I suspect that wherever the D1 database is placed, is likely somewhere that's like too far to actually benefit from being at the edge. This is again a result of being at low volume. So I'd probably use a different database like SuperBase or Torso, for example, that can be like distributed, but I have more control over where it gets distributed. So what I ended up doing is I added, caching using something called zone cache, which is a feature that we use, at Zoopla, which is essentially an edge, caching solution. and basically what I decided to do was cache based on the prefix, so like the first few, digits of the latitude and longitude. what you do actually is when you remove like the specificity of the longitude and latitude or truncated essentially, you're creating like a geo map. so there's like a square, of Earth essentially that's that all the locations are going to be cached for. So as long as you have like over 10 locations in your results for that area, that's probably going to be the same 10 results for everywhere within that geographic block essentially. and this cache is done at the edge. I can show you the code, if you're interested, it's all open source. So I'll mention that later on. But essentially I took all these results for this, Geographic square, cached at the edge, and this results in calls that take a lot less time, about 70 milliseconds on average. And, that's pretty good actually. that's faster than what Capital One was doing on their API. I remember, if you recall, it took them about 89 milliseconds. I'm gonna go and refresh this guy again. Clear this out. pop this over to the side and the first call again, that's going to take a while. It took 1. 84 seconds to make a second call that's cached and look at that 25 milliseconds. Let's make a handful more called. Look at that. All of these are like sub 40 milliseconds in terms of like response. So that's half of what Cloudflare or say half of what Capital One took to respond with the same number of results. And that's all a result of caching. So as you can see, Your data's location matters a lot when it comes to where, if you should be using an API at the edge or have your gateway at the edge or data at the edge, your data store, what you choose to use for it, and it matters a lot basically in terms of your first call, but for subsequent calls, caching at the edge is almost always a better solution, so much better in fact that this is a mind boggling fast, I should probably fix my slides. I'd say around 40 milliseconds, which is like crazy fast for an API response. So that's it for my demo. And that's it for you know why you should run your APIs at the edge. If you have any questions, feel free to connect with me on LinkedIn. This is the QR code. You can scan over here. If you have any, if you're interested in seeing the code for how I wrote this API, I've open sourced all of it. Please send me a DM again on LinkedIn if you're interested, and I'll see if there's some sort of resources for the conference I can share this in later. You can also take a look at the slides, I think, after the fact, in case you're interested to get links to the code or any of the images that I had. Thank you so much for attending my talk, and enjoy the rest of your conference. See ya.