Kube-Native ETL at Scale: Optimizing PySpark + Airflow Workflows in Cloud-Native Environments

Abstract

Discover how to scale PySpark + Airflow ETL in Kubernetes-native environments! Learn real-world tuning tactics that cut latency by 50%, enabled 1,000+ DAGs/day, and supercharged cloud-native orchestration for data platforms at scale.