Hello everyone. Good morning. Good evening. My name is Murli Kdi Manou. Welcome to Con 42 Site level Engineering 2025. Today we'll talk about scalable interconnect strategies for GPU accelerated HPC clusters, GPU Interconnect Challenge. The problem, GPU computational capacity has evolved exponentially outpacing interconnect bandwidth improvements by three to four times per hardware generation. This widening gap creates severe performance bottleneck in large scale distributor systems limiting the effective throughput of multi GPU computations. So how does it impact in modern HPC applications? Interconnect latency. A network congestion can consume 30 to 50% of the total execution time. The substantial overhead persist even in meticulously optimized computational computation intensity workloads significantly reducing overall system efficiency. So what do we need to do next? Generation solutions must address three fundamental challenges, bandwidth saturation at extreme scale. Topology aware routing inefficiencies and the substantial synchronization overhead of collective operations across thousands of distributed GPUs. Let's look at some of the traditional interconnect limitations. First, bandwidth saturation. GPUs generate data at rates that overwhelm network capacity causing. Memory buffer congestion and forcing computational pipelines to stall across distributed compute nodes. The second routing inefficiencies. Static routing protocols cannot adapt to realtime network congestion, creating traffic bottlenecks, enforcing data through suboptimal paths during high throughput workloads. Third synchronization overhead. Multi GPU collective operations require precise barrier synchronization, where even nanosecond latency variations compound exponentially, severely degrading performance as systems scale to thousands of nodes. Let's dive into some of the top policy innovations that are. In the marketplace First Factory architecture. This network topology architecture delivers non-blocking communication with full BI section and bandwidth and deterministic latency scales effectively to thousands of nodes, but requires exponentially increasing switch count at higher radis. 3D Taurus Network topology. This topology implements a mesh like structure with wrapped around connections, minimizing I wiring complexity while maintaining low hop counts. Optimized for nearest neighbor communication patterns. Common in physics simulations, dragonfly network topology. This topology leverages. Hierarchical organization with high erratics routers to minimize network diameter and cable length achieves near optimal trade off between local and global bandwidth while reducing the cost and power consumption. Let's understand RDMA direct access efficiency. RDMA enables CPU bypass, which enables direct memory to memory transfers across the network fabric without CPU intervention, which dramatically re reducing crossing overhead and system resource consumption. This also enables low latency. Achieves up to 55% reduction in end-to-end communication latency, allowing near instantaneous data sharing between distributed GP nodes in computing intensive applications. This also provides higher efficiency, which delivers 97% protocol efficiency for medium sized message transfers, virtually eliminating network overhead. And maximizing effective bandwidth utilization across the cluster. Let's look at some case studies. First one, frontier supercomputer, 20 point 24.8% gain was realized by dynamic adaptive routing algorithms significantly outperformed traditional static approaches. In high traffic simulation workloads. This also uses slingshot interconnect proprietary congestion control mechanisms dynamically adjust packet priorities and pathways to maintain optical optimal throughput under extreme loads. This also employs dragon fry to policy as we reviewed previous slides. Enhanced network architecture delivers near uniform global bandwidth distribution with minimal diameter, reducing worst case latency by 37%. Let's look at another case study, which is Nvidia Seline. This uses Infinity Band, HDR. This is revolutionary to 200 gigabit per second interconnect. Between compute nodes with full bisection bandwidth architecture, eliminating network congestion and ensuring seamless parallel communication. This also implements R-D-A-R-D-M-A, which delivers exceptional 97% protocol efficiency for medium-sized data transfers while slashing end-to-end latency by up to 55%. Dramatically outperforming conventional networking approaches. This also implements NCCL optimization. Precision engineered GPU to GPU collective operations maximize throughput with advanced topology aware algorithms that intelligently adopt communication patterns based on workload demands. Okay. With that, let's get into some of the software optimization strategies that are being used in the industry. First one, adaptive routing intelligently reconfigures network pathways in real time based on traffic analysis, reducing congestion by up to 40% and delivering sub microsecond latency across complex workloads. Okay. Topology aware algorithms, sophisticated communication frameworks that precisely map data, exchange patterns to the physical network architecture, reducing network diameter, traversals by 60%, and MA minimizing cross switch traffic overhead. N-C-C-L-Q, advanced customization of GPU collective operations. That orchestrates communication patterns with nanosecond precision, eliminating t transfers and achieving near the theoretical bandwidth utilization load, balancing sophisticated traffic distribution algorithms that dynamically allocate bandwidth across multiple pathways, preventing resource contention and maintaining consistent 95% plus throughput efficiency. Under extreme computational demands future interconnect technologies that are evolving integrated network processing units. These are purpose built silicon accelerate packet processing and routing operations at line rate, complete off offloading of communication protocols from GPUs. This liberates computational resources for core workloads. Photonic interconnects, silicone photonics enables multi tera data transfers using wavelength, using multiplexing power consumption decreases by 65% compared to the electrical interconnects while sub nanosecond latencies become achievable. Third in package integration, high bandwidth network interfaces. Co-pack with the GPU substrate using advanced chip plate architectures, drastically reduced signal paths, minimize propagation delays, and unlock unprecedented GPU to network through output. Let's look at some of the performance metrics and benchmarks. Traditional ethernet maximum is 12.5 gigabytes per second. End to end latency. 10 microseconds. Peak performance utilization, 65%. InfiniBand, HDR. Maximum dropout is 25 gigabytes per second. End-to-end latency 3.5 microseconds. Peak performance utilization is 85% NVIDIA and NVLink. Maximum throughput is 50 gigabytes per second. End-to-end latency, 1.8 microseconds. Peak performance utilization. 93% future photonics. Maximum throughput is a hundred gigabytes per second end-to-end latency 0.5 microseconds. Peak performance utilization up to 98%. Let's review key takeaways and next steps. Analyze your application communication patterns. Conduct comprehensive workload profiling to identify critical data, movement, bottlenecks and communication hotspots. Select appropriate network topology, strategically align infrastructure investments with specific application requirements to maximize performance to cost ratio. Implement software optimizations, fine tune collective communication operations specifically for your network architecture. To eliminate redundant data transfers, pre prepare for emerging technologies, develop modular systems and abstraction layers that can seamlessly incorporate next generation interconnect advancements. Thank you. If you have any questions, yeah, please reach out to me on my LinkedIn.