Hi, welcome to Con for Machine Learning 2022 conference. I am Karan Singh, senior principal architect and developer evangelist at Rodhat. In this talk, I will explain how to deploy and use edge to core machine learning data pipelines on kubernetes for a smart city use case. We will also learn how to automate everything using ansible it. Let's get started. Let's start with the business use case. A business requirement. We need to build an app which or a solution. It's not a single app, it's a collection of apps. We need to build a tax stack that should help us reduce some congestion. And we need to charge vehicles some kind of fee as they drive into a city area. So we have chosen an area in London called as ultra low emission zone, in which there is a special charge that the vehicle needs to pay because the authorities wants to protect or because of environmental reasons they want not all vehicles should just enter into the city. So yeah, reduce some congestion, reduce some pollution, charge some dirty vehicle fees. If the vehicle does not meets emission standards, we want to apply more charges so that the owner will not better the city very frequently. And maybe a third use case would be, hey, I want to locate a wanted vehicle, right, which the officials can use the system for. So that's a business, very high level business requirement. And we're going to walk together on this and touch upon how we can capture all these things. So that's the techsu I was talking about. I'm going to show you lots of tech, lots of, lots of tools. Open source and red hat tools, lots of tool. And each of them have their specific reason why they exist on this solution. Right. We don't want to over engineer, but yeah, there is a special place for Kafka, for Ceph, for superset, for Grafana, right? For databases, obviously. Kubernetes is the heart, the beating heart for all the solution. Ansible is going to be the magic through which we're going to create some tech soup on this. So yes, before we go into the Morse presentation mode I'm using to go and show you, let's deployed, let's deploy this thing right now and let's use ansible to do it for us. And while Ansible will do the heavy lifting in the background, I'm going to take you over to a few more journey, like what I'm doing in this actual implementation. Meanwhile, the system will be on its toes. So what you guys are seeing here is I have my terminal opened up here, I've already logged in to my openshift Cli. On the other side we have Openshift console through which we're going to see how things operate. So step one, there is nothing on the system right now. It's a vanilla, not vanilla, but it's an empty openshift cluster. And that's the first workload, the smart city workload we want to deploy on this cluster. So starting with starting the basics, create a new project, right? New project OC, new project called a smart City. You want to run this? Okay, we are into the project and the CLI is smart. It have already added me to the project. Next I'm going to run an Ansible playbook, which I'm going to explain you in a few slides. So bear with me. This is going to do a lot of things in the background. This process will take close to ten to 15 minutes. Meanwhile, we'll go and explore how the overall solution looks like. So we're going to run ansible. So those of you who are familiar with Ansible, Ansible playbook is the command to do it. I'm going to run it from local because I have the connection established on my local machine. And then this is the main master playbook we're going to run against. So I'm going to hit this. And meanwhile this is running. Let me pull up my project, smart city. The project is here. And just go to my favorite section, workloads and parts. So we're going to see lots of stuff coming up, right. So coming back to my deck, back into presentation mode, system will take some time to set up. Let's understand, let's understand, meanwhile what is happening under the covers. So from the solution design point of view, okay, we have this city of London ultra low emission zone. We have all these cameras installed across various toll locations across the city. And the use case is that each toll location that we're calling as an edge, right. Each edge location will recognize the vehicle, the passing vehicle, and detect its license plate. And through machine learning models we're going to grab the string that the model detects in real time from all these edge locations and append some metadata to it. Like okay, what is the timestamp through which this image was captured? What is the geolat long? So that we can do all sorts of amazing stuff once we have a lot of data into the system. Okay, but the data is being generated at the edge and there are multiple edge locations here. How we can move the data to the central data center, to the central system where we can do lots of compute and analytics, do some fee calculation on the data that we have collected, apply some more business logic like, hey, apply some dirty vehicle, dirty charges to the vehicle and notify the officials in case if the system finds out, a lost vehicle. Right. So remember our use case that I was explaining before. So that's a very high level design. Part one, in part two. Great. We are live capturing data across from all the edge locations onto the central core location. We have done some business analytics in there. But wait a minute, we are capturing a lot of data. What should we do with this data? Well, we should retrain the model because obviously machine learning, the model has to be updated as we learn about the data, as we collect more data. Right. So we need to store vehicle images, license plate images and license plate strings into the system. Right. And then retrain the model. We have a model already deployed, the edge, retrain the model, improve its prediction accuracy, and then deploy the same model or the new version of the model across to multiple edge locations. So Openshift helps to do that. And tools on top of Openshift helps you to build a solution like this where you have multiple, few hundreds to thousands of edge locations. You just need a system that provides you the right tool to do this kind of stuff. So that's part two of the solution design. Right? Deploying the model at the edge. Let's come back and understand how these things work. Right. So this is more kind of a 1000 foot view of the system. We have multiple edge locations running Openshift environment. It could be fat edge where there could be a three node openshift cluster or maybe a very thin location where they just need a single node, a single node openshift. Right. That's really a thing right now. So as the video streams produce images, live images from the system, the images will go into a license plate recognition model. This model will then extract out the areas of the image where it detects a potential license plate, and then it will basically give this subset of the image to an OCR model, optical character recognition model, which will read the letters from the number plate. Right. It's pretty standard. And then once we have, once you have data onto the edge, what we'll use, we'll use Kafka. Kafka is the go to technology to handle this kind of workload. So we'll use Kafka, Kafka producer, and then Kafka producer will produce some messages onto the local Kafka cluster running on the edge. But now here's the magic. We have lots of edge locations and we need to move the data. We need to build this loosely coupled system that should handle the network disruptions, right? So Kafka does it pretty well for us. All the events, all the metadata will be stored on the local Kafka edge clusters. And then asynchronously it will move the data from edge to core using a technology or using a feature called as Kafka Mirror maker, which will move the data from one edge or maybe thousands of edge locations onto the core central data point, which is Kafka again. So we are capturing the data onto the central Kafka cluster. And once we have the data onto the central Kafka subsystem, fantastic. We can do lots of things with it. For example, we can build our own Kafka consumers, maybe a python go or Quarkus apps that can read and tap into the Kafka topic and build some business logics, do some wanted vehicle notifications or else for long term preserving the data of kafka. Because remember, this is all governmental data, right? We want to store data historically, correct. So we need some persistent layer that can move data that is coming to Kafka. Move into an object storage. Because object storage is autonomous storage, correct. You just store it. It's simple to use and it is cheap, it's low cost. So we'll use something called a secr which can help secure is by the way, a Kafka consumer which will tap into the Kafka topic, read the data from Kafka, store that onto object storage bucket. And here we are using Ceph object storage, which is the choice. And next, once we have data onto object storage, we can build, analyzing and reporting, real time analytics and reporting using tools provided by open data hub like Superset, Apache Superset, Open Grafana, Starbus. So Starbus is a very amazing technology. It can allow you to write and build distributed queries across heterogeneous data sources. Like you can write a SQL query that's very powerful, which can go and read data from a SQL database, and at the same time you can join it using data that is living on the object storage s three interfaces. So this is the power of superset that provides you to have this kind of environments, which is a very powerful engine. And then Superset is basically the dashboard and reporting part of it, which will show you in real time the reports. And Grafana. Obviously, to manage this system, we would need cool developers like you, as well as some operations people who want to keep an eye Eagle's eye onto their dashboard. And they need some real time dashboard and Grafana is like the best choice, at least in my opinion, because I love Grafana. So this is the overall solution that the ansible automation under the covers is doing for us, right? So what ansible is doing for us, let's understand. It's installing and setting up database for us. It's installing and setting up Kafka clusters because this is a demo right now. So Ansible is doing deploying an Kafka edge cluster as well as a Kafka core cluster on the same openshift environment that we have. And then ansible going to deploy multiple microservices. So one of them would be license plate recognition microservice, which includes using build config builds and then from git source to images and deploying that into deployment config and launching the pod and creating the service and exposing it through the route. So Ansible is actually doing a lot of work. Similarly, like LPR, Ansible is setting up microservice for events, microservices for image server, and microservice for load generator. Because I don't have a real camera hanging around and looking at the street. But yeah, so we have a load generator for the sake of this demo and then setting up secure, which will move the data into the system. So this is all microservice setup that Ansible will do for us. And then setting up open data hub which is the collection of tools which will help us do analytics in real time. And then finally the Grafana dashboard setup. So Ansible is really doing the heavy lifting here. While I'm speaking and talking to you right the moment the system is doing the job for me and for all the enthusiasts out there. What all modules I'm using to make this happen, it is just these eight modules I'm using to get my system up and running from zero to 100. So there's a community model called as KS, which basically uses Openshift client, Python client library client tool, I think library, which it will call up, and it has full access to all sorts of openshift objects. So ansible can just tap onto the KS module and boom, you can just do whatever you want, you can do with an OC client, right? Similarly, by the way, OC client does crud, it will do and change the objects while Ks info another module, it will go and read out basically all the get commands that you do, like OC get or kubectl get. Ks info will do the more or less the same stuff. We are using set facts because we have to manage the state and dynamically update the inventories and have some variables that we wanted to capture into the system. I'm running some commands because there are certain things which are still complex and I don't have a module for. So I'm like supercept, superset, data updations. I'm using some commands for it. I'm using copy modules to copy images from my data store to s three and then adding host to the inventory s three, sync to move the data and a pause just for sake of completion. Like I'm pausing it just so that system can get stable. So yeah, look at this. I mean, I'm just using the eight modules to do a lot of heavy lifting using ansible. And Ansible is a really powerful tool to do this kind of automation for you. Okay, let's now quickly check how does ansible, Ks module or ansible usage will look like you will write something like this. Okay, Ks module, the state is present and I need to apply multiple yaml files like, okay, create a secret, create a deployment config, create a service, create image stream, create build config. So ks module and ansible will make sure you are applying these yaml files like you do in your OC create or OC apply minus f command right and into the right namespace. So that's what Ks module will do. It's pretty simple to use and then KTas info is doing the other way. It's basically getting the data from the kube API. Okay, tell me the secret and the name of the secret is this. And register the value in a postgreSQl register variable. And then later on you set facts to pull out secrets like database username, passwords and name that you can later on use in your playbook. So this is how you will create these ansible playbooks for your lovely Pacman apps. Okay, and all of the code is available onto the git repository. It's open. Go and look it out. Check out. Happy to get some prs. If you guys interested, Edson will share some links to the link and credits to my friend Chris Bloom who is an awesome guy who helped us with lot of ansible automation around that you're going to see. Okay, let me go and get back to my system. Okay, so the system is still doing the job. You can see that ansible, I'm going to scroll up a little bit and show you. It's not very user friendly, but since I'm using this in a very narrow window. But yeah, look at this. Deploying databases deployed Kafka clusters and deployed Kafka drop APIs and then all sorts of stuff that I explained to you. So system is still doing the job. So bear with the system. Let me see. Yes, so I'm into all projects. Let me go to smart city. Project smart city is here. And yes, let me see if I'm getting any failures. No, there's no failure at the moment. So things are. So at this point, Ansible is importing some data source into superset. So at this point it has done lots of setup already. We are into the superset part of the equation. Meanwhile, while this is getting ready, I have another environment where I was pretty sure that this is going to take a little bit of time. So I'm going to switch my window. It's another environment which is completely set up. So I'm not cheating here, I guess, or you guys can let me know if I'm cheating. So just to save time here. So this is how the end result will look like you're going to deploy lots of containers, lots of pods, lots of services. And Ansible is basically automating and doing all the heavy lifting here. So this is my developer view. If you are a fan of administration view, go to the right project from here, smart city, and then you will see. Okay, great. I have lots of pods running into the system, I guess. Yeah, 36 pods. These are jobs. But yeah, 36 pods up and running at the moment right now, which is making happen this demo. So I'll open one of my route, which is the dashboard, Grafana dashboard for the bird's eye view of the city. So as I mentioned before, this is city of London. And all these black boxes here are camera, and we have this generator which is generating images or like simulating vehicles which are passing from the station in real time. We are capturing the count of the vehicle. We are detecting the vehicle, last known vehicle from the system. We are detecting its license plate, which is the optical character recognition, and the license plate plate recognition model of the vehicle and who is the owner. So by the way, these names are just made up names. And here's a nice graph which can tell you this is a rich data that we are collecting. Okay, please tell me, out of the city, what are the top stations we have in the city? Right? So station one a is very popular because there are so many vehicles passing from the system. So you know what, these are some business metrics and data that you can take up. I'm going to real quick show you the cpu chart because I like this, because it's a gif and it's pretty awesome to see how this looks like. Okay, as I mentioned before, we have edge Openshift. Edge openshift core. We have Kafka and inferencing happening on the edge. And data flows from the edge via mirror maker onto the score Kafka cluster, and via secure data, move to ceph self object storage. And then later we use Starburst and PostgreSQL and other tools to do some dashboarding around it, which looks like this. So this is the view for your managers and your key stakeholders. This is a reporting view that they will see. Okay, so far we have done, like 215,000 pounds of collection, toll fee collection, and 100,000 pounds of pollution fees. And 45,000 vehicles have passed from the city. Right? Again, this is all generated data, similar data. However, just to give you an idea that what you can do with these tools once they all work in tandem, and then we have these nice panels in superset which you can use, adjust it according to your business case, and you can get very precise metrics about your business. Okay, station number 5201 is getting 22% of the total traffic of the city, which means we need to do something here, right? Maybe add one more lane, maybe build a superpath here, right? And then again, vehicle, so not very interesting. But anyways, what type of vehicles are more popular in the city? Okay, Nissans are popular, or Audi R eight are popular? We can debate on that. And some table panels that. Okay, tell me, my top consumers, who needs to pay a lot of money to the government? Okay, so Suzanne King needs to pay, like, whatever, 15,000 pounds. Oh, why? That's a lot of money to the government. But anyways, this is a fake data, and you get an idea, right. This is the view for your managers that they can enjoy. And openshift and tools and projects running on openshift can make it happen. And all the nifty Kafka lovers here, I have seen that. Kafka. I love Kafka. Kafka is great. And so I'm using this kafdrop, which is a nice, very nifty utility. It's an open source project that you can tap into. You'll deploy it. You have the code in my repo, so, yes, it will help you to check out Kafka messages in real time. No need for Kafka cat. Instead, just plug into this. It's an open source project. When you open this LPR, and then you will quickly show me the messages. So this is real time messages coming onto the LPR. Topic at the core data center right through this URL. So when I expand this I'm getting events like okay, timestamp and event id and vehicle license plate and detection is successful and on which station we are doing the detection those kind of data. I think we definitely have another deployment of this. Let me see where we are on my deployment. Okay, wonderful. Fantastic. So this worked at least demo God is with me. So ansible is completed. It has took like 29, oh yeah, 15 minutes. It has taken 15 minutes to deploy the entire setup that we have shown. By the way, we have already gone through all these things that you want to show. The only thing I want to show you here is if I go to network and see routes. There are so many routes in here. Odh. OdH is a great piece of solution. I'm just using two components of ODF. That's why we don't have a lot of components. But check out ODF. ODF is pretty cool. And then I want to show you just last thing I've shown you core, right? I've shown you core. This is core Kafka cluster. I just want to show you my edge Kafka cluster as well. So these are the cluster deployed at the edge. The topic name is LPR license plate decode recognition. I'm going to view messages and I'll do quick view. Okay, so the point here is that we are capturing the message at the edge via mirror maker. Oh, let me go two steps back. We are doing inferencing at the edge, detecting the vehicle, capturing the license plates in real time, putting that strings onto Kafka topic from the edge, moving the data to the core. And then once the data is in the core we are doing analysis on top of the data. Analysis like this. Analysis like this. So this is what I have to show you guys. Coming back to my deck. This ansible openshift and lot of the tools makes this happen. Okay, so I think I am done with my prepared content and this brings us to the end of the presentation. Thank you so much for your time. I hope this is what's branding. Have a nice day.