Hello everyone. Welcome to real time personalization at scale, neural ranking systems and operational breakthroughs. I'm Vedant Agarwal, Senior Software Engineer working in Search. Today, I will share how advanced neural ranking systems enable personalized recommendations to drive both operational efficiency and real business impact. In this session, our agenda is organized on around several key topics. First, we will cover our breakthroughs in ranking, which includes both candidate generation and precision re ranking. Next, we will address latency challenges, discussing how we can achieve sub 50 millisecond response times, which is a crucial factor for ensuring a smooth user experience. We will then explore the real time constraints of processing millions of interactions concurrently. And finally, we will review the business impact of these solutions and look at emerging trends that will shape future initiatives. This overview is meant to provide you with a clear roadmap for the presentation ahead. Real time personalization introduces several technical challenges. First, data inconsistencies and noise are common, as user clickstreams and transaction logs often contain variability that can affect model accuracy. Second, scaling becomes a critical concern when processing millions of concurrent interactions while maintaining system responsiveness. Third, model drift poses an ongoing issue As user behavior evolves, consider, for example, a sudden shift in seasonal trends, our models must adapt quickly to maintain relevance. These challenges underscore the need for agile and robust solutions capable of processing high volume, real time data effectively. To overcome these challenges, there is something as a two tier neural ranking architecture. The first tier, L1, focuses on candidate generation using embedding based indexing. Here, user behaviors and item attributes are projected into a shared high dimensional space, allowing for rapid filtering of a vast number of items. The second tier, L2, handles precision re ranking using advanced sequence modeling techniques. such as LSTMs or transformers to integrate both historical data and real time signals. This decoupled approach enables us to optimize speed in the first stage while ensuring that the final recommendations are highly personalized. For example, while L1 may retrieve a broad set of items from a database of over a million entities in a matter of milliseconds, L2 refines this set to match the user's current context accurately. In our L1 candidate generation process, we achieve significant performance gains through embedding based indexing. This method projects both user behavior and item characteristics into a shared feature space, allowing us to employ approximate nearest neighbor search technique. For instance, our system can retrieve a broad set of relevant candidate items in under 10 milliseconds, even when querying a corpus of more than a million items. This rapid pre filtering is crucial as it reduces the computational load for the L2 ranking stage, ensuring that The system remains both fast and scalable. The efficiency of L1 is a foundational operational breakthrough that enables us to deliver highly responsive and personalized recommendations. Now, we move on to L2 stage, which focuses on precision re ranking. In this layer, we employ advanced sequence modeling techniques using models such as transformers or LSTMs to capture the sequential dependencies in user data. Music playing This approach allows us to integrate both real time session information and long term user history effectively. For example, by incorporating attention mechanisms, the system can assign higher weight to recent clicks compared to older ones, ensuring that the most relevant signals are prioritized. This dynamic re ranking process, which adjusts in real time to the user's context, can lead to a significant improvement in recommendation accuracy, with an observed increase in click through rates by approximately 20 percent as shown in studies. Next, let us discuss latency breakthroughs, and how we can handle real time constraints. The latest systems are engineered to achieve sub 50 millisecond end to end processing response times, ensuring a fast result generation from user action to recommendation delivery. We can balance the inherent complexity of our models with the need for quick inference By simplifying certain architectural components without sacrificing quality, for instance, by implementing optimized inference pipelines with asynchronous processing and caching strategies, we can reduce latency from around 70 milliseconds down to 45 milliseconds. Additionally, system scales dynamically under load. Thanks to auto scaling and load balancing, which are continuously monitored in real time to maintain consistent performance at the heart of operations lies a robust infrastructure and dynamic feature engineering process, we can utilize real time data pipelines. Employing tools like Kafka and Apache Flink to continuously ingest user clicks and purchase data, ensuring that our models receive instant updates. Microservices architectures containerized using Docker and orchestrated by Kubernetes can enable us to maintain modular scalability and fault isolation. In parallel, feature engineering process dynamic It generates features from live data, such as current session attributes, and applies advanced transformations like normalization, feature crosses, and depend meetings. Observability can be maintained through tools like Prometheus and Grafana, which provide automated alerts to detect and address performance anomalies promptly. Going on to the next slide, deploying and updating the system seamlessly is critical. Which is why we can have robust deployment strategies. We can maintain regular monitoring through defined KPIs and automated anomaly alerts to catch performance deviations early. Automated retraining sessions can be scheduled to address model drift, ensuring that our models remain up to date with evolving user behavior. Our deployment processes, including blue green deployments and rolling updates can help us achieve zero done down times during rollouts. For example, we often can use canary releases via Kubernetes to safely test new updates before a full rollout. Additionally, A B testing allows us to gradually roll out features and assess their real time impact with these pipelines, tightly integrated with our CI CD tools. Finally, let's consider the business impact of these operational breakthroughs by refining ranking accuracy and reducing latency. We can directly improve the matching of products to user intent, which in turn can increase conversion rates. Personalized recommendations have also led to enhanced user engagement reflected in longer sessions and improved click through rates. In practice, we have observed quantifiable outcomes such as 15 percent increase in conversion rates and a 25 percent improvement in CTR in these different studies. These metrics are tracked through integrated analytics dashboards that we can continuously monitor and adjust our business strategies to meet our objectives. This is where the technical part of the presentation ends. But looking ahead, we can focus on several emerging trends that will shape the next generation of personalization. One of these trends is multimodal personalization, where we integrate text, image, and video data to gain richer insights. For example, by combining visual cues from product images with descriptive text, our system can be made, can make more nuanced, Recommendations. We can also explore real time federated learning, a decentralized approach to model training that not only enhances privacy, but also reduces latency by keeping data local. Additionally, we can enhance our neural ranking by adopting next generation deep learning models to push accuracy even further. Hybrid approaches are under investigation as well, where rule based systems are combined with AI. offering improved interpretability without sacrificing performance. The long term vision should be to continuously adapt to market trends and evolving user behaviors supported by ongoing R& D aimed at integrating state of the art model architectures. In conclusion, Today, we have reviewed operational breakthroughs in real time personalization. We discussed how ranking innovations, both the rapid candidate generation in L1 and precision ranking in L2, work together with latency optimizations and real time scalability to produce a robust recommendation system. These technical improvements have translated into measurable business outcomes such as improved conversion rates, higher user engagement, and better customer retention. As we look to the future, emerging trends and next generation personalization initiatives will continue to drive our evolution. Finally, thank you for joining me on this presentation. Feel free to connect with me on LinkedIn.