Conf42 Machine Learning 2025 - Online

- premiere 5PM GMT

Serverless AI: How Modern Architecture Slashed Costs and Doubled ML Model Deployment Speed

Video size:

Abstract

Discover how leading enterprises slashed ML costs by 40% while doubling deployment speed using serverless architecture. I’ll reveal the exact patterns that transformed AI delivery across industries, with practical steps you can implement immediately.

Summary

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Hi everyone, and thanks for joining us today. I'm excited to talk about a powerful safety in machine learning operations, and one that's making a real impact across the industry as wireless architecture. Over the next few minutes, I will show you how this modern approach to infrastructure is not just a buzzword, it's delivering serious results. Organizations are stressing operational cost while doubling the speed at which they'll deploy the machine learning models. We'll work you through the core principles of wireless AI share findings from or cost industry research and give you the practical framework that has been like proven in real order, real world implementation. So you can take these ideas and apply them right away in your organizations. Before we dive in. Let's have a quick intro. I'm KU and I have spent past few years working at intersections of ai, cloud architecture, enterprise transformations, machine learning, et cetera. You can find my contact details and LinkedIn here on the slide. So I am always open to connecting and answering the questions and continuing the conversations after the sessions. So feel free to reach out to me. Let's look at the real impact. Serverless machine learning is having organizations ing serverless architecture are seeing a huge improvement starting with a 65% reduction in infrastructure management over it. That means less time spent on maintenance and more time focused on innovations. On average, they're also seeing like a 40% in cost savings compared to traditional deployment methods. But perhaps most exciting models, deployments has jumped three and put times annually, and teams are able to iterate and see models faster than ever before. And it is not just a, backend improvements. Users engagement is up to 28% since in products that have implemented serverless AI and showing a get link between the architecture and end user impact what this tells us is simple serverless doesn't just make these, make the things more efficient. It unlocks scales and speed strategic focus for data teams. These slides really highlights one of the most dramatic benefits of going serverless deployment speed. Traditionally deploying a machine learning models could take around two to three weeks with the team stacks in the cycles of provisioning infrastructure. And configuring the environments and setting up the manual, scaling with a serverless transitions, those steps has rings to about three to five days. And team focus on the things like the configuring the API gateway, containerizing functions and integrating with the cloud provider. But the real magic happens once you, you are really serverless and deployments are dropped to just two to four hours on an average. So everything is automated. There's no infrastructure to manage and scaling happens in Stanley. So this kind of speeds is not just nice to have. It's a really competitive age and in industry where rapid iterations and AI driven features define the market leaders. So the ability to see models in hours instead of weeks is a game changer. These slides focus on the financial side. This bar chart breaks down exactly where server list machine learning earn its repetitions for cost savings. On average, organizations are seeing like a 40% reductions in cost compared to traditional machine learning deployments, and it's not just a one area. The savings came from poor key fronts like infrastructure maintenance scalings, and DevOps. With serverless, you would not need to verbal poisoning Hardware maintenance is minimum and scaling is automatic, and DevOps efforts are dropped like dramatically, and by using the tool. What really stand out here is the paper use model. So it's a perfect fit for the machine learning where the workloads often spikes around training or deployment, but remain ideal. In between serverless ensures that you are only paying for what you use, which means no wasted spend during during the downtime and full powers when you need it. Great. And these slides really showcases like one of the core strengths of serverless for machine learning. One of the most powerful features of the serverless machine learning is elastic scaling, and the pyramids is break down. Break it down. I. And the top, you have the peak performance. Serverless systems can automatically handle thousands of concurrent predictions request without skipping a bit and be of that like a dynamic scalings and the ability to instantly adjust to realtime traffic changes. So there is no mini manual tunings or the planning needed. Then we have the predictive capacity, some systems, even the village machine learning themselves to anticipate usage pattern and allocate resources ahead of demand. Finally, it's all sits on a cloud. Foundations distributed across the multiple availability zones for resilience and the performance. What that means, like in practice, it's simple and organizations no longer have to over poison infrastructure just in case of a traffic spike. And during real world like 10 time traffic search serverless machine learning systems maintained stable and low latency responses and delivering consistent consistent AI performance when it matters most. These slides connect the technical improvements to real business outcomes. Now let's talk about the operational efficiencies. Because serverless is not just about the savings times or money, it's also about doing more with the same team. Organizations are moved to the serverless. Were able to deploy 3.4 times more models annually. That's a massive leap in, in output and it changed how fast team can respond to new opportunity and challenges. So looking at the pie chart here, deploying velocity improved by 72%, giving team the ability to seep updates and improvements faster. Model diversity increased to build in support for multiple frameworks like a Inflow, py Tots, and others. The continuous deployment has become the norm with the automated CI I CT pipelines, reducing the friction from experimentation and through the protections. And then finally, more rapid. It since unlocked the new innovation potentials leading to smarter and more adaptive PI features, and it's pay off organizations saw like a 28% boost in user engagement after implementing these serverless powered AI features. And that's a direct line from architecture to business impact. This is a great slide actually for grounding the strategy in a real world architecture. So here we are looking at a layered architectural pattern that emerged as a best practice for serverless machine learning deployments. Starting with the data ions layer, everything is the event driven from the streaming data to automatic features extraction and optimized storage. These NCOs raw data is immediately usable for the machine learning. Next, the model training layers use really uses like a thermal compute meaning compute resource spin up, like only when it need and, we can get on demand, GPU, access and hyper parameter during functions and ability to scale scale out the distributed training jobs without managing the infrastructure. Then we move to the in finance layer. So where we train models are containerized and start behind the behind an API gateway. We can have automatic scaling in points and model problem worsening and testing, casting for high frequency queries, all fully managed. Finally, the monitoring layer and the ties it all together with performance in ES and model drifting detections and even automated returning triggers so your models stay accurate and relevant over the time. Structure allows teams to scale and optimize each part of the machine learning lifecycles independently, while still benefiting from the flexibility and the simplicity of serverless computing. This slide ties everything together by showing the real world impacts across the sector. Let's wrap up. The core content by looking at how these serverless machine learning strategies are being applied across the different industries, each with their own unique challenges and goals. So in financial services, like serverless, machine learning is a power is powering like a four detection system that reduce pulse positive by 34%. Even as transactions volumes increase five times. These models runs in real time and continuously adapting to new for better, no time, downtime, and no batch deal. So in healthcare, serverless functions are being used for medical imaging imaging analysis, what used to take days. Can now be done in a minute. So these systems scales automatically to meet the patient's volume and the remain remain hipaa compliant, which is like health insurance. Portability and accountability Act compliant even with the massive data sets and distributed processing. And in a rural companies are using like a serverless machine learning to deliver hyper hyper-personalized recommendations. And these have been like a 23% boost in average order values. Thanks to the model that updates continuously in real time. This also supports season seasonal forecastings and the inventory optimization. So the key takeaway here is while the underling server serverless principles are the same, implementing the strategies must align with the industry specific needs whether it is compliant. And the speed and the customer experience. These slides focus on how to refine the performance. So performance optimizations is a key of getting like the most out of serverless machine learning, especially as the workload skills. So let's dive into some technicals some techniques that have been proven to make the big difference and faster the model compresses. So by using this technique like quantization pounding and knowledge distillations, so you can reduce like the model size up to 60 to 80%. This helps you to fit your models within the serverless deployment constant without sacrificing the performance. Then we have the cold start mitigation. So serverless environments often faced like latency users where when functions have to warm up and to solve this you can implement the warm pooling strategies and preload carbon models and manage functions concurrency to minimize these impacts. Memory optimizations is another big win. Right sizing, memory allocations based on the model needs to ensure that you are not over poisoning or underutilizing resources. So pr that with adaptive batch, adaptive batching to maximize throughputs. And you have what a lean and efficient setup. Finally casting strategy by using like a multilevel casting at the API gateway functions and the database layer, he would reduce redundant computations and is spliting up the inference times and the lowering loads. So now what the, is like companies use using these strategies reported as 65% improvement in response type and additional 25% cost deductions beyond the initial serverless savings. The base results comes when you combine several of this techniques and tailored to the specific models and usage pattern in your environment. These slides provide a step by step roadmap. So our deliveries would be clear and structures and successful server list lesson learning adoption doesn't mean starting from the scratch. So the key smooth migrations is integrating serverless capabilities with your existing machine learning workflows. And it can be done in three main phases. The faster is the assessment phase. This is where you evaluate your current machine learning workloads to identify the candidates for serverless migrations. Focus on models with variable inference pattern or those that require the frequent updates. So you will want to perform like cos benefit analysis to determine the best workloads for migrations taking into the account usage pattern and scaling needs and any technical constant. Once you have assessed and it's the time to start the pilot implementation. So begin by migrating the non-critical models to minimize the risk. During this phase, you will refactor the model for the serverless deployments and establish the performance baseline and implement the CICD pipelines for the automated deployment. Key task here is to include like the containerizing, your models and configuring like cloud services and setting up the monitoring to track the performance. The scale and optimize phase comes after the pilot. Once you have seen the success with the initial model, expand to additional workloads. This phase is all about like fine tuning, optimizing the performance, and creating the inter internal based practices as you move like higher value models into the product sense and document everything to yeah, across the team and ensure those smooth transit smooth transitions. And our research source like that following this phased approach, reduce like migrations risk and after it's time to value in fact 87% of pilot project made into the full production within just three months. To wrap it up, let's go over the key takeaway and the next steps. So first, when we are starting like your our server list machine learning journey focus on the interference workload with the variable traffic pattern, these are often like the best candidates to see immediate 20 feed, such as improved cost efficiency and the scalability. Next make your the implementation focus specific to your models, prioritizing, like the optimizations techniques that align with the characteristics of the, of your workloads, whether it's model compressions cold start mitigations, or the casting. Lastly for the long-term success, develop a comprehensive serverless machine learning roadmap. This will guide your migrations and NCO. The serverless principles are open into your long-term, long-term AI strategy. In summary, serverless architecture are is levelizing like how organizations deploy and skills ai. It can, it can reduce the cost by 40%, and organizations are deploying 3.4 times more models annually. The elastic scaling handles the unpredictable traffic and empowers data science teams to focus on what they do based on the innovations. So to get started, identify a few high impact interv interference workload from a small cross-functional team, and develop proof of concept using the architectural pattern we have discussed today. And measure your result against the established baseline and use those inside to build your comprehensive serverless machine learning roadmap. Thank you all for your time and at today, and I hope you found the insights on several latest machine learning valuable and that you are excited to explore how this approach can transform your organizations. If you have any questions or you would like to discuss further, feel free to reach out. To me after the presentation. I am happy to connect and my LinkedIn details are already in the very beginning of the slides so you can, and thanks again. I look forward to hearing you about your serverless journeys, and thank you.
...

Tarun Kumar Chatterjee

.NET Senior Lead Developer @ Presidio

Tarun Kumar Chatterjee's LinkedIn account



Join the community!

Learn for free, join the best tech learning community for a price of a pumpkin latte.

Annual
Monthly
Newsletter
$ 0 /mo

Event notifications, weekly newsletter

Delayed access to all content

Immediate access to Keynotes & Panels

Community
$ 8.34 /mo

Immediate access to all content

Courses, quizes & certificates

Community chats

Join the community (7 day free trial)