Conf42 Kube Native 2025 - Online

- premiere 5PM GMT

Scaling AI-Driven Financial Services: Kubernetes Orchestration for Ethical ML at Scale

Video size:

Abstract

Discover how Kubernetes transforms ethical AI in finance! Learn cloud-native patterns for scaling responsible ML systems that democratize financial access while maintaining regulatory compliance. Real-world architectures for fraud detection, risk assessment & financial inclusion at global scale.

Summary

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Good morning. Good afternoon, everybody. My name is jd. I'm here to talk about one of the significant transformation happening in the financial industry. Let's get into the presentation. My screen is visible scaling AI driven financial services. So simplifying. We are, we will be deep diving how we use a technology like Kubernetes to build and manage the powerful systems, those arely driven and at massive scale. The core challenge we are facing is not just making AI work, but making it work responsibly for everyone everywhere. So we are in the midst of financial revelation. The world, the financial world is changing at the speed. And AI is the engine. It's a driving factor. This transformation it's creating an incredible new opportunities for first time. We have tools accurately assist the risk and provide the services to unbanked population that has not been historically and. Under the governance or was excluded from the financial system. And the biggest example we have seen back in India in 2014 when Han accounts were being opened at the scale. But this power brings complexity, right? How to scale the fraud detection systems handle millions of transactions every minute. How to ensure the credit scoring models are fair across dozens of countries and cultures, right? This is where the cloud native services comes in. They provide the architecture foundation we need to build the AI systems on that are not only powerful, but are scalable and responsible and ethical. Today's journey. When we look at it like first, if we look at the first challenge that we could define here in the roadmap is scaling, of course and scaling the AI solutions. The second we will explore here the container solutions like Kubernetes orchestration patterns, and that forms the backbone of. The model system. And third, we will focus on ethical integration, discussing how we embed the governance into our technology, workflow, et cetera. And finally, it comes to putting this all together in the real world implementation and see how this concepts bring together the best practices at the scale. Let's skip cracking. So the core challenge as we discussed is scaling. The financial market is incredible dynamic. All AI system must scale instantly to handle sudden market volatility or transaction spikes during the peak hours while maintaining the response times measurably in microseconds or milliseconds. That's a big challenge. The traditional monolithic system simply cannot scale or keep on. Imagine a single, massive application that is trying to handle fraud detection at a global sales like Thanksgiving or Black Friday, right? And while simultaneously running the credit algorithms for millions of users, it becomes a bottleneck. Elasticity just isn't there, right? So what's the solution for that? The solution to this problem is scaling scaling, using content orchestration. With Kubernetes as a leading platform, it's built on key principle First is microservice architecture. We here we break down the massive monolithic systems into a smaller, independent, scalable services. Think of it like a giant entangled ball of yarn and set for netting instead of that, an individual falls, which can do better, right? Second is dynamic scaling Kubernetes can automatically add or remove resources in real time. It's like a horizontal scaling, upscaling and downscaling possible at the peak hours. Imagine at the start of trading session in NASDAQ or New York market opens, it's sudden spike in the market required massive resources, but then eventually it goes down and we need to have a system that withstands all those load and. Dynamically scales itself and that all is possible using Kubernetes. And the third is multi-region deployment which allows us to distribute our services across the global. It's critical because of globalization. And it is critical for performance, resilience and meeting the regional compliance policies, which we will be deep diving in, in, in a while. Serving the unbanked. Now let's connect the technology that we just discussed back to people, right? This scalable cloud native is what makes it possible to serve previously unreached population. We can build mobile first credit scoring models for emerging markets. We can process alternate data like mobile phone, using utility and payments or user patterns, et cetera, to create a financial profile for those who traditionally don't have a credit history. And we can use containerized, NLP and natural language processing models to offer support for dozens of languages and making it more reachable to the people. You of a different culture, different language background, et cetera. This is a financial inclusion in action, a power of smart architecture, and this is exactly what we need, right? Ethical challenges. If we look at it like as our AI system expands, any small underlying bias in our model gets amplified across millions of users. Think of it, a model that unfairly penalize a certain demography in one city, could now. Do it across the entire continent, countries and this distributed environment, environments, ethical government stops being just a design consideration. It becomes critical. It's like now you need it. You must have it like and think, right? Infrastructure concerns should be managed consistently and constantly. So let's look into that, how we embed our ethics. So how do we build the governance into our infrastructure? We can use cloud native principles. What is that like? First is policy as a code, we embed our ethical rules and fairness guidelines directly into our deployment PowerPoints. This means the more that model doesn't meet, our fairness criteria can be automatically blocked. Even before reaching the production, second is automated mon monitoring. We build continuous bias detection, fairness, checks directly into our monitoring checks. It is like having a smoke detector, which is always on for biases, right? And third is audit trails. It's very vital to have an immutable unchangeable logs that can be for every model decision. Governance action taken. And this provides a very high level of transparency and even meets the regulatory requirements, which are critical in banking or financial industry technical implementation. Plat, and let's get bit more technical here. These governance principles are implemented through specific patterns in. Model lifecycle management. We use CICD pipelines to manage our models from deployment to production, from staging to productions and transition. This includes GitHubs Jenkins, like pipelines to clear versioning of a model to have clear versioning running automatic fairness test before they. Testing staging environment before the reach to production. And we use cannery deployment to roll new models to small subset of the users to monitor the bias, fairness, and limit the back to limited people, and then fully release it into the production, right? This is tried into observability. We tie this all steps into what is called as continuous monitoring, right? And observability. We don't just track CPU and memory utilization. That's a traditional way of doing things. But we build the custom metrics that can track the model fairness and and in real time, and. We can use distributor tracking to understand the entire path of decision making as well, and set up an automated alert if the biasness metrics crosses certain predefined thresholds. So multi judiciary compliance as in a financial, global, financial world, right? You have to deal with the complex web of international laws. Kubernetes provides a tool to manage this for region data re residency. It's one of the very critical, or very much a strict law of land, right? We can use the rule to ensure that the data for specific countries stays within the specific countries. Like for example, Germany, Zurich they want their data to store geographically on the same servers. Residing on their own land as very much possible for regulatory reduction. We can configure our Kubernetes services that are for regional laws and policies without having to rebuild the entire stuff from scratch. That's a massive leap actually. And. Cross border coordination. Now having a global system, financial organizations do need some coordination in what do you say, cooperations or collaboration between two regions. And this technology like service mesh can manage secure communication between different regions and we can achieve the coordination of cross border while abiding to all the rules and regulations of the country. Let's see. Put together all of this and see the real world case study, a global financial fraud detection system. The challenge is massive processing, say a hundred million transactions a day across more than 30 countries with with with response times, which is under. A hundred milliseconds, all while ensuring that the models are fair and not being flagged for certain user groups of Biasness. That's a massive challenge. The solution architecture is exactly what we have been discussing. Kuber clusters are deployed in each major region. The real time model serves the Easter service like mesh to have a secure communication between two different regions. The the model monitoring systems like Prometheus or Grapha or AppDynamics can handle the biasness and track the model's, fairness and other customer metrics in real time. Finally a ops workflow or Jenkins workflows that automates the deployment with built-in Gates to check ethical validations. Before the new models are being live performance resilience, of course, in the financial organizations, the system must be rock solid. This architecture provides high availability through multiple zones, deployments, ensuring near perfect uptime. It delivers elastic scaling. We have discussed that for handling transactional spikes or without overspending on infrastructure. And it has security integrated built in every layer from network policies to the port security standards to protect the institute financial information or financial data, et cetera, et cetera, et cetera. These are all critical factors that needs to be considered in performance and re part of your architecture key implementation strategy. If you remember just three things from today, right? Let's let them be like three keys or three strategies, right? Start with governance now. Don't treat ethics as an afterthought, as we already discussed, how badly it can impact, right? Build your fairness frameworks before you scale because the problem can replicate to global population and it can become a very massive, big problem and amplify itself. Monitor continuously. That's the second key, right? Deploy comprehensive observability for both technical parameters, fairness, and all of the custom parameters that you find in must have for the entire framework. Automate everything. Think of it as. As using GitHubs or Jenkins and policy as a code is enforcing governance and consistency across each and every environment. It becomes more reproducible and replicable also. All of these things will just bring a lot of efficiency when you want to deploy these things or this architecture in different geographical region. Think about it. Without automation, it's highly impossible to achieve all the leaf that we discussed in a given time. What's the path forward, right? So the cloud native technologies like Banes are no longer just about efficiency. They are essential infrastructure for scaling, AI driven financial services responsible. By integrating ethical governance directly into our technical workflows we can finally build systems that serves global population while maintaining the transparency, fairness that the future of finance demands. And we have seen them in detail, right? To leave you with one thought, right? The future of financial inclusion depends upon the ability of scale AI systems. There are both technically robust and equally sound. With this, I would like to thank you all for attending and giving your valuable time. I am happy to answer all your questions. You can contact me with my email on the screen or with my LinkedIn account. Thank you.
...

Jaydeep Taralkar

Executive Director @ UBS

Jaydeep Taralkar's LinkedIn account



Join the community!

Learn for free, join the best tech learning community

Newsletter
$ 0 /mo

Event notifications, weekly newsletter

Access to all content