Conf42: Python 2021


Event-driven applications: Apache Kafka and Python

Francesco Tisiot
Developer Advocate @

Francesco Tisiot's LinkedIn account Francesco Tisiot's twitter account

What’s better than a pizza example to show how Python and Apache Kafka, a streaming platform, work together to enable reliable real-time data integration for your event-driven application? Lets dig into problems Kafka is solving, its Python libraries and prebuilt connectors together!

Code and data go together like tomato and basil; not many applications work without moving data in some way. As our applications modernise and evolve to become more event-driven, the requirements for data are changing. In this session we will explore Apache Kafka, a data streaming platform, to enable reliable real-time data integration for your applications. We will look at the types of problems that Kafka is best at solving, and show how to use it in your own applications. Whether you have a new application or are looking to upgrade an existing one, this session includes advice on adding Kafka using the Python libraries and includes code examples (with bonus discussion of pizza toppings) to use. With Kafka in place, many things are possible so this session also introduces Kafka Connect, a selection of pre-built connectors that you can use to route events between systems and integrate with other tools. This session is recommended for engineers and architects whose applications are ready for next-level data abilities.

Apache Kafka is becoming the de-facto standard for streaming applications with many companies basing on Kafka their whole data pipeline. In micro-services contexts Apache Kafka runs the fundamental role of decoupling information’s producers and consumers thus making sure data pipelines are scalable and reliable. I’m a Developer advocate at, leader in the managed Apache Kafka sector, spending most of my time in developer’s shoes building reproducible examples of data pipelines. I’ve been blogging and speaking around the world since 2014 about various aspects of the Data lifecycle: from producing applications, analytics, visualisation and streaming platforms.

Awesome tech events for

Priority access to all content

Community Discord

Exclusive promotions and giveaways