Conf42 Platform Engineering 2025 - Online

- premiere 5PM GMT

Building Resilient AI Pricing Platforms: Engineering Real-Time Decision Systems at Scale

Video size:

Abstract

Learn how to build millisecond-response AI pricing platforms that handle millions of decisions daily. We’ll cover edge computing, ML pipelines, circuit breakers, and MLOps patterns from real retail systems serving global customers at scale.

Summary

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Hello everyone. I'm Amish Sahu, vice President of Pricing and Data Analytics and xometry. Today we're going to talk about building resilient AI pricing platforms and how we can make real time decisioning at scale. As we know the modern retail landscape has changed dramatically. And these days pricing decisions are becoming becoming increasingly complex and time sensitive. And today we are going to talk about how we can build sophisticated AI powered systems that are just price in real time basis. Based on market conditions competitor data or competitor trends and customer behavior patterns. These dynamic pricing platforms are represent some of the more complex challenging architecture are technology ecosystems today. So we'll briefly discuss that and cover some of the aspects around it as we know. With pricing, the stakes are high. You can make a direct impact on revenue. The requirement requirements are extremely complex and we are dealing with massive data volumes from different data sources. We have to watch out the performance and the latency to maintain consistent performance. Reliability is another key factor here. You have to make sure the systems are reliable across geos, geographies and also we are maintaining the regulatory requirements to ensure that we are balancing the revenue impact the complexity of the system, and with the reliability across geographies. So a little bit about the infrastructure architecture for low latency pricing. As there are few aspects we'll cover here. One is the edge computing, which is essentially making sure you are distributing a computing resources across different regions across pricing systems. So that you can reduce response time while maintaining consistency across different markets. You have to make sure when customers are on your site, you're not taking too long to show them pricing through instant coating you're making sure that you are pulling the information from different sources or systems in a way that is fast and also that is accurate. You need to have a container orchestration where we want to make sure there is automatic scaling based on the demand. And as the demand, as the volume goes up, your system should scale seamlessly to provide the right information as we talked about the complexity across, dealing with different data systems. Consistency. Data consistency is another challenge. You need to have the real time inventory or sourcing information. You need to have the competitor pricing data. You need to have customer behavior, PA patterns. All those things end up getting together. So we have to make sure the data is consistent. And last but not the least, you have to think about network optimization. The content delivery networks that optimize for API responses dedicated network connections between data centers and how we can make have intelligent routing behind it to make sure we are minimizing the latency here. When we talk about the architecture, we we need to discuss about data pipeline engineering. As we talked about the complexity of the data and accessing data or different types of data from diverse kind of systems. You need a constant stream processing where you're continuously ingesting and transforming the data. Across the systems as I say, whether it's competitor data or inventory data or supply data or customer behavior interactions, all these things needs to be processed in the right way. And you need to have real time validation for that. So implementing real time data quality checks. Of course you cannot add more latency or processing delays. You need to be smart about how to detect anomaly. How you make sure that you are doing the real time validation without compromising on the performance or the speed. Of course you have to think about time-based features that can pull from historical data and in a way that. You are whatever net new or incremental data you're getting, you are basing your processing or calculation based on the incremental data in a way that you're not taking too long to process it. So building key here is that you need to build the robust data pipelines that can handle this complexity while maintaining real time performance. A lot of, so robustness in the technical architecture and operational processes go a long way here. The other thing we talk about is in pricing. This is most of, as we think about pricing in today's world, most of it is AI driven pricing. So how do you make sure that, you are deploying your machine learning models in a way that the models are don't have latency issues, but at the same time, you're providing the high accuracy across different type of market conditions or, across all kind of the data aggregation or data processing we are doing. So we have to. Make sure you are allocating the memory correctly to all the, all these models for a fast response time. How we can be strategic about where you can cache the prediction or where you have to make it real time inferences for edge cases. So essentially as you say, compartmentalizing the modeling with that. Some of the common scenarios, you are caching the prediction, but for some of the edge cases, you have the intelligence or you have you, you can run the models to provide the right information. When you think about pricing, you have to continuously experiment. So how we have the right experimentation set up, whether it's AB testing or multi bandit or contextual bandits, how we can be smart about experimentation and how we can have the infrastructure to support the experimentation to provide the right customer outcome and financial outcome. And as we've. See most of the cases, these decisions on pricing, are they need to evolve over time. There will be cases where you we may end up implementing something that doesn't work out as planned. So we need to have, or at least to a system issue. In those cases, we have to have automated rollback triggers based on KPIs or some of the guardrails we have set up. So in a way, we the systems we have or architecture needs to be robust enough to a sup to get the right data b, persist the data correctly and see support advanced models and machine learning models during the process. When you think about machine learning, model management we always have to think about how we can make them more effective and efficient. As we know, most of these models are combination of multiple models. There are models behind models. We have to continuously evaluate how efficiently we can combine predictions from this. Different models how we can use them selectively depending on their strengths how we control for the biases or late and any shortcomings to the model to make sure that at an overall level we are providing the. Right information with right accuracy. Version control is key as we are evolving these models in and continuously, improving our predictions, improving the way we provide price informations. We have to make sure we are versioning. We have version control so that you can roll back if needed, or you can refer back if you want to see how the new model is performing compared to the old model. Continuous performance monitoring is key. As. We evolve the models as pricing has to be nimble and agile. We have to make sure you're continuously monitoring the performance in a holistic way. Which when I say holistic way, we have to think about customer outcomes, financial outcomes. All those things need to be looked at together and all those performance monitoring need to happen in real time accounting for all the different data sources. Last but not the least here is with this models, we need continuous training. So how we can make our infrastructure robust enough to ensure that the models are getting the right feedback and trained on a regular basis so that we are making these predictions better and better over time. With this models and the complex data architecture, we have to keep in mind scalability and performance optimization. As we've discussed with the demand volume or as the volume of data grows, we need autoscaling so that we can capture the right input for the models. You have to balance the the volume, incoming volume and the competition need. Additionally, you have to measure where in the databases you can optimize to make sure that we are able to manage heavy workloads all the complex queries or complex calls, we are able to drive them efficiently. Just so it goes long way in terms of you, you not only just need the right and accurate. Predictions. You also have to make sure that you, we have the ability to scale and optimize our performance. The other areas will be caching and resource management. Again multi-layered caching with aggressive caching can improve your response times. As we discussed a couple of slides ago, where, what. Common scenarios we can cache so that you are not going through the rigorous prediction every time. So how do we make sure not just the prediction and also input date has a is fresh and how we can cache effectively to make sure that we are using the right combination of streamlining and also edge case optimization to drive the right performance. Some of the scaling patterns here will be non-linear depending on product catalog size, market complexity, and algorithm sophistication. So we have to account for that as well. And additionally, as we say, the performance monitoring. Traditional performance application performance monitoring tools may not capture the unique characteristics of pricing systems. So we have to think about right customization that can track prediction, latency, model performance, and business impact to provide better insights. Then we can talk about different reliability and fault tolerances. So there are different approaches here. The circuit breaker patterns when external data become una unavailable or internal services experience, degraded performance circuit breakers can prevent by isolating the problematic areas. We can think about graceful degradation as well. Strategy might involve serving cash prices using simplified price ang pricing algorithms or falling back to default pricing rules. When we have issues with the accessing the AI part, pricing services similar data application is key because, synchronous replication for critical data while using asynchronous replication for less time sensitive information can balance performance and reliability requirements here. So data replication is key in case we have accessibility issues and we want to make sure we have a fallback fail safe option here. As we talk about system architecture, disaster recovery and health monitoring are key. Again we should have the ability to recover if there are issues in one geography or in one of the systems. So how we can. Consider both data restoration and model state reconstruction, as well as the time required to rebuild caches and restore full system performance, comprehensive health checks. Health checks must go beyond simple connectivity tests. We have to make sure the pricing system is working as a whole. So comprehensive checks that not just look at the connectivity, but also verify model loading prediction quality, and also provide real time insight to into the system health. And system performance. I think that's the key here. So it has to be comprehensive. Additionally because we're across geographies, across different systems, we have customer data that we, we need to use for pricing, predictions, security and compliance are key here. How we are ensuring data protection. How we are training the models on the data which essentially talks about federated learning, how we ensure regional compliance when you are across geographies. Dealing with data for Europe, European customers, or dealing with data for north American or Asian customers every, depending on the region. We have compliance requirements. We have to make sure we are following the regulations. And with vast amount of data comes a lot of responsibility. How we are making sure we have the right access for the right people and making sure that we are going with the role-based access that provides the right level of granularity for the right, right stakeholders. That goes a long way. And having the right authorization framework will also go a long way. With this we talk a little bit about the operational excellence and monitoring. So again, we are, for operational excellence, we need to have multidimensional monitoring, which needs to be comprehensive. Whether it's technical metrics, model performance, or financial metrics, we need to monitor all of them. Incident responses. What are the automated mitigation strategies? What are the escalation procedures? What are the cross-functional response teams? How we do retro and post incident analysis. All those things we need to account for. With that, we talked to a bit about how the architecture or how we can build a AI pricing system. As we know in the world of ai in pricing, things are continually continuously evolving. So there are future trends that we need to prepare for and we need to account for as things evolve. Edge AI capabilities, quantum computing or real time personalization are the ones where things are moving very fast. We have to make sure we are evolving our architecture and systems and to handle all those complexity that get introduced as we make progress in all those different aspects. So finally building, as we talked about it, building a robust, resilient AI pricing platform is one of the most challenging engineering problems today. As we talked about, its direct revenue impact, how decision is made by considering data from different systems. Whether customer data or financial data or competitor data or market data. All those things lead to very nuanced decision making. That's where our systems, our architecture they need to balance the competing demands for performance, reliability, scalability, and security, while making sure we are adapting to the changing business requirements. As the retail landscape or ma marketplace landscape evolve the AI pricing platforms will play a free role here and they need to evolve to address some of the challenges we discussed during this presentation. Thank you so much. Appreciate your time and it is a pleasure sharing some of the thoughts around this topic that I'm deeply passionate about. Thank you.
...

Amaresh Sahoo

Vice President, Pricing, Sourcing & Analytics @ Xometry

Amaresh Sahoo's LinkedIn account



Join the community!

Learn for free, join the best tech learning community

Newsletter
$ 0 /mo

Event notifications, weekly newsletter

Access to all content