Conf42: Cloud Native 2025

Cloud-Native ML Infrastructure: Building Resilient Apache Spark Clusters on Kubernetes for AI Workloads

The convergence of AI/ML workloads with cloud-native infrastructure presents unique challenges in scalability, resource utilization, and operational complexity. This talk demonstrates how to architect production-grade Apache Spark clusters on Kubernetes that specifically cater to the demands of modern AI/ML applications while adhering to cloud-native principles. Key Topics - Implementing cloud-...