Hey guys, welcome to Con 42. Thank you for joining this session. My name is she Reya and I serve as a senior engineer engineering manager at Truco. Over the last several years, I had the privilege of leading engineering teams responsible for building and scaling large distributed systems. These are the kinds of platforms that power millions of users and they come with some unique engineering challenges. One lesson I've learned is that scaling isn't just about throwing more servers at a problem, anyone can provision more compute or more databases, but true scale comes from making systems resilient, adaptive, and importantly simple for developers to work with. That's really the heart of today's stock. We'll explore how caching. Something often treated as a minor optimization can actually be transformed into a self-healing intelligent infrastructure component. We'll see how that shift reduces latency, improves reliability, and enables growth without overwhelming engineering teams. So let's begin by looking at the kinds of challenges that make this shift necessary. So let's start with the pain points that almost every large scale system runs into. As platforms grow, demand scales faster than most. Infrastructure can keep up with. Databases that once seem to rock solid, begin hitting connection limits during peak traffic. APIs that were fast under small loads suddenly start chiming out unpredictably and, sure. Many of you have been in situations where an engineering team spends days or even weeks firefighting, rushing to patch database overloads, or trying to chase down why latency spikes during a big traffic event and what's happening. In those moments, developers are distracted from building new features because they're stuck keeping the lights on tradition. Traditional caching helps a little. But it's limited. They have static tls those time to leave settings. Al data either expires too soon leading to unnecessary re computation, or it stays around too long serving scale results. Scaling is usually manually manual and reactive. Someone notices load building up and spins up more nodes and observable pretty is poor. Engineers don't have the data they need until a user complains. This patterns repeat across companies, industries, and technologies. They highlight why we need caching and infrastructure that can actually adapt on its own. So what does a better approach look like? The vision is to build systems that are self-healing. That adapt, recover, and scale without manual intervention? There are five guiding principles I want to highlight. First one is self-service platforms. Developers shouldn't need to be distributed systems experts. If a product team wants to add caching to an EPI, it should be as simple as a config change or a notation, not VIX of deep engineering. Second. Intelligent systems traffic patterns shift constantly. Maybe a product launch in Europe causes traffic to surge at 3:00 AM Local time systems should notice adept and rebalance caching strategies automatically self-healing mechanisms. Failure happens. A cash node goes down a replica legs. A region experiences network issues instead of engineers waking up at 2:00 AM The system should promote replicas, reroute traffic, or rebuild state from logs without human input. Fourth one is comprehensive observability. Self-healing doesn't mean blind automation teams still need reach, visibility dashboards, metrics and recommendations. So they understand what's happening. Fifth one is effortless scaling growth should not require a proportional increase in operational effort. If user traffic doubles, engineers shouldn't suddenly be working. Doublers, the system should handle the growth gracefully. This vision transforms infrastructure from something BRI and reactive into something resilient and adaptive. Now let's move from vision into architecture. How do we actually design a caching platform that supports this? A key insight is that caching works best as a layered strategy. At the foundation you have database caches reducing repeated query costs. About that distributed caches act as shared key value stores accessible across services. So for example, a Redis is an example of distributed cache. Then application level caches hold hot data right where it's needed in memory. Finally, edge caches deliver content closest to the user, improving global performance. But layering isn't enough by itself. The design is held together by four architectural principles. First one is service mesh integration. The caching layer integrates with the service mesh for automatic discovery, routing, retries, and circuit breaking. This services degrade gracefully under stress. The second is event driven architecture. Caches are updated by streams, not polling. That means a sink, warming, real time currency, and even reconstruction state from event log. Third is directory based ency at a massive scale invalidating caches. Cache is reliably is hard. Directory based currency protocols help ensures accuracy across thousands of nodes. Layered strategy by spreading caching responsibilities across multiple tiers, no single layer becomes a bottleneck. So what's powerful here is that this principles combine to make caching not just an optimization layer, but resilient backbone for platform engineering. So static rules only go so far. This is where machine learning steps in with ML caching becomes adaptive. So let's look at some of the. ML techniques. So first one is predictive cash warming. Instead of waiting for a request, ML models anticipate demand base on historical usage, popularity spikes, and even time of day patterns. This reduces cold start delays. Second is adaptive, TTL management static expirations brittle, mL dynamically adjusts TTLs based on data volatility and excess patterns, balancing freshness with efficiency. Third is anomaly detection. ML identifies the unusual traffic potential cash poisoning attempts or performance degradations before they escalate. So here the impact is clear. Caching moves from being reactive to being proactive. Predictive and self tuning. So how do we make this work in practice? There are three main implementation patterns. First is automated pipelines. Caching infrastructure should be defined declarative. Validated through GitHub's workflows and rolled out with zero downtime. This ensures consistency and reduces risk. Second is developer friendly APIs abstract away the complexity. Complexity. Developers should be able to annotate methods or endpoints and instantly benefit from caching strategies without needing to know the internals. Third is self healing mechanisms. Systems detect failures within milliseconds, promote replicas reroute traffic seamlessly, and reconstruct lost state from logs. When these patterns come together, caching becomes resilient, consistent, and developer accessible. Let's talk about observability and monitoring. None of this works if we can't see what's happening. Observability is what gives developers confidence. So if you can build dashboards that could track cash or cash hit or mh ratios latency distributions like P 50, P 95, P 99 memory utilization and eviction rates error rates and timeouts. But then more than raw metrics, observability needs insights. For example detecting that cache hit rates dropped sharply after the deployment, or recommending that caching a particular endpoint could cut latency significantly. Observability, if you think about it, observability transforms raw data into actionable guidance, and it's what enables true self feeling. What's the out outcome of all this work? The impact of self filling caching is transformative. It basically provides latency reduction users experience faster response times across the board. Database load reduction query volume, drop dramatically. Freeing capacity, third one is high. Each ratios caches maintain efficiently even during peak surges and global consistency. Geo distributed caches delivered reliable performance worldwide. In other words, caching shifts from being a small optimization to being a strategic enabler for scale and performance. So let's zoom in. How machine learning actually powers caching. There are different ML techniques. And briefly describe few ml techniques here. So first one is neural networks. So neural networks learn patterns in data access and predict which items to preload. We could also apply reinforcement learning. What it means is it could continuously refine eviction strategies to maximize hit ratio, hit ratios. We could also use clustering what it means. So clustering means group workloads or users for targeted cash warming. The benefit includes higher hit ratio than static approaches. Reduce memory usage via smarter eviction. Better search prediction during traffic spikes cost efficiency through optimized resource use. Now this refrains, this sort of, this reframes caching as a continuous optimization problem powered by machine learning. Now let's talk about the edge where the caching gets even trickier. Extending caching to the edge introduces unique challenges. First one is currency at scale managed. And this can be managed with eventual consistency. Vector locks. Vector clocks, and read repair mechanisms. The second challenges limited resources, edge notes have less capacity. So dynamic sizing intelligent eviction and. Compressions are essential here. Third one, third challenge that we have often seen is network partitions. Edge caches must continue serving users even when disconnected and then reconcile when reconnected. So by solving this caching ensures performance is strong everywhere, not just in core data centers. So technology alone doesn't guarantee success. Practices matter. So best practices include self-service infrastructure. What it means is developers can provision caching resources with one click run simulations and configure without deep expertise. The second is operational excellence. Automating repetitive work, applying predictive maintenance and minimizing to toil. Together these practices ensure caching platforms are reliable, trusted, and developer friendly. The next slide talks about the culture cultural transformation. So culture is the multiplier that determines success. Technology alone. Technology alone won't scale unless culture evolves with it. So what does it mean? First is culture, which empower developers, give them ownership of caching strategies with tools that simplify adoption. The second is shared responsibility. Reliability isn't siloed ops task. It's shared across teams. The third is data driven culture. Use metrics and feedback loops to guide continuous improvement. The now what is the payoff? The payoff is faster delivery, lower cost, and a better user experience. Culture is what unlocks the full potential of technology. So what lessons can we draw from building systems at scale? Some of the lessons include developer experience comes first. Simplicity and good communication matter more than feature overload. Incremental migration is essential. What it means is support legacy systems while enabling smooth transitions. Third is observability is non-negotiable. Metrics, validate assumptions and guide decisions. Third is expect challenges. Technical hurdles like hotkey. Technical challenges like hot keys are solvable, but organizational resistance and skill gaps require just as much focus. And these lessons are universal across large scale systems. Finally let's look at where this field is headed. Near term trends include stronger ML models, graph ql, caching optimization, and automated capacity management. Midterm. Midterm, we will see innovations like serverless cache functions. Realtime cash analytics novel validation methods and post quantum security approaches long term, the vision is AI driven optimization, autonomous operations and intent base infrastructure systems that understand what you want, not just what you configure. The TRA trajectory is clear. Caching and infrastructure are becoming more intelligent, adaptive, and autonomous. Let's wrap up with the big picture. Self hailing caching isn't just about performance. It's about reducing latency, lowering database load, maintaining reliability globally, and empowering developers. Caching has evolved from being side optimization into a strategic foundation for resilience and scalability. If there's one takeaway from today, it's this investing in self-healing cashing. Unlocks faster systems, happier developers, and sustainable growth. Thank you.