Hi folks. I'm Chu. welcome to my talk on the Power of Cilium with EVPF and let's jump right in. So first, let's talk about what is EVPF? in layman terms, EVPF is basically a tool that you can use to attach functions to system calls. So basically. Like we have the send and receive system calls that are used to send and receive, data from the machine. so if you attach functions to that, then in that case, if any application calls those functions, or those system calls, your function will automatically get executed. And you can do all sorts of things there, like when you can access the data that is being sent. To the system calls or you can access the data structures in the system calls, or you can perform any specific function that you wanted to when the system calls are hit and EBPF is actually an extension or an upgrade on top of BPF. That was a tool that worked in a very similar way, but, the only difference. Was that it was only attachable to user space, functions. And it could only be used to filter packets and not perform the advanced functions that we can using EVPF. and the biggest, benefit of using EVPF is its, performance. So basically you can attach. All the probes that you want with minimum performance overhead, because these EPF probes are really lightweight and they attach very efficiently to the system calls, and they definitely don't hinder your, the a, the performance of your application. so if high performance is one of the priorities for their application, you, EBPF is definitely an option that you can look for. and in terms of what is cilium? Cilium is basically used to write the user space code for getting all the data and everything that we have done in EBVF happens in the kernel space. So to get that data in the user space, you need to write user space code. So for that we use cilium, and cilium is not the only option out there. we have a lot of options in c plus Python and also in Rust. the reason that we went with Cilium was that first of all, our rest of the application was written in Go and, Go is definitely one of the better languages for observability applications. So we, wanted to stick with Go and also, cilium and Go in general has a really active and helpful community that you can leverage if you run into any sort of problems with it. And that is what we definitely did and we got a lot of help from that. apart from that, cilium is a really active project and it has active maintainers that are always, ready to help or take a look at any of the issues that you are having. So basically, your development becomes much faster. While using it. And that is what we experienced. But also there are other options in rust, c plus and Python. I, and like one of the reasons that we did not go for them is, first because rust is, definitely more, memory safe compared to the other languages, but it has a steep learning curve and. There is not a lot of support for a T eight. we were not sure if that would be the ideal way forward if we couldn't get, a lot of support because E-E-E-B-P-F itself is a really recent technology, so that too has, really, limited support and docs, so we. Wanted to go for something with a bit more support and, c plus, was getting like really low le level for us, and that is what we did not want to do. So we ended up sticking with cilium. But if you have other requirements or you do not agree with these reasons, there are these bunch of options that you can go for. So for Rust, you have ia. And since, it is in rust, it obviously offers better memory safety compared to the other alternatives that we have in go and c plus. But obviously, as I mentioned, it has a more steeper learning curve and, the support is also not as much as cilium. And for Python, you have B, C, CB. C. C provides you all the libraries that you need for, writing your user space code, for getting the data. And you also have the BPF that you can, in c plus and also PCC and Lake VPF are actually two of the more. Popular, most popular, languages used for not languages. Their frameworks used for, writing kernel space coding, A BPF. But one major difference between them is that late BPF runs on the idea of compile once, use everywhere. So basically if you have. Written your source files. You just need to compile it once, and then you can run that EPF program anywhere as long as you have the object files with you. But that is not the case with BC, C because you will need to compile it every time that you need to run your EVPF application. The problem becomes that you will need to include your source file in the main, repository of your project, which could be a deal breaker for a lot of people. And that is also one of the reasons that we went for lib BPF. so we could just compile it once on our end and we could just use the object files in the. in our main repo and that would get the job done for us. So it basically depends on your needs and what you prefer. So I just, wanted to talk about user space and the kernel space a bit more. So the user space is basically where your application runs and where all the user side, Handling happens. So basically everything that you can run and interact with is running in the user space. And it obviously has very limited access to system resources as it is on a very high level as compared to the kernel space, but it is where your users interact. With their application. so that's why it is important. And the kernel space is really more low level compared to the user space, and it has direct access to obviously all your hardware. And it is, it typically has really high performance and it. Perform all the low level, operations for your application. so here we are using EVPF as a bridge so that, we can get the data from the kernel space and then use it in the user space so that we can take advantage of both the kernel space and the user space in our case, and we are able to show. The user, the most low level data that is possible to be shown. so here is the, c an example of C code that we had used. So basically after you have, Retain your kernel space code. You just need to compile it. And that would generate objects for you in cilium. And then you basically need to attach the probes to all the calls that you had. let me first explain this using a diagram. This is by Pixie and this explains it really well. So suppose you have an application that is making. Encrypted calls to a website, right? So in that case, it'll first call the open SSL, library to encrypt its data and then it'll call the send and receive system calls to send and, receive that data. So in this case, if you wanted to access the data before it gets encrypted, and if you attached a probe to the send system call, that would not work because in the data is actually getting encrypted in the TLS, library and not the send and receive system calls. Basically after the application calls the DLS library, it encrypts the data using the SSL read and the SSL right functions that it has. And then after the data is encrypted, it is sent to the send and receive system calls. So if you try to access. The data here, all you are going to get is gibberish. So for that, you would need to attach probes to the SSL Read and the SSL right functions because that is where the encryption is happening. So you can access the data before that encryption happens, and you can do all sorts of things with that. So here that is what we are doing. So we open. Per prefer to check for all the events on those functions. That is the SL lead and the SL right functions. And then we, attach probes to those functions like. Here, and here we are using a U probe because, this TLS library is in the user space and not in the kernel space. So in this case, we are attaching a U probe and if we were to attach, probes on send, receive, or any of the system calls, we would be using K probes. So in this case, we attach a probe each at the ending and the starting of the function. So here we have a probe at the starting of the SSL right Function and one at the ending of the SSL right function. And similarly, we attach probes with the SSL Read entry and the exit, and we keep polling for any data that we get. And then we can do all sorts of things with that data. So to demonstrate this, I also have a demo. This is, pixie demo that I'll be using for, showing this. So what we have here is essentially what we talked about. We have a client and a server application that are basically establishing a TLS connection with each other, and then they are sending encrypted data to each other. And we are going to try to get that data using our EBPF program. So l let me just show you how the application works first. lemme start the server here and start the client here. So I have logged the PID, first of all to Use it in our tracer. And this is basically what is happening. It is the server is sending it some secrets and the client is sending the server some secrets and they have established an SSL connection with each other. so let's look at the A BBF code that we have for this. So if we go to the tracer. Here we have linked it to the open SSL library that we have in our machine That is. So here we have attached it to our library. That is the lab SSL library for Linux, and we have attached the probes on the SSL, right? And the SSL rate functions. And we have essentially attached two probes. One at the entry of that lab, of that function, and one at the exit of that function. and here at the end, we are just going to be polling these functions for any kind of data that we get. So as you can see in this infinite while loop, we just keep polling for the data as long as we, as long as the EPPF probes are attached to it. And in terms of functions we have. The functions right here. If you see, so here we have the actual functions that are using the processing. So first let's take a look at the SSL Read and the SSL right function. So here it is, getting the data, using this specific, register called. Two. And we are, and we have basically created a hash map that, that would map the data to a specific TGID. And that is how we are going to be storing it and processing it. And at the end here, he just submit this event. So that it could be logged by our tracer. So here, as you can see, we have functions for both SSL Read and SSL, right? And for the entry as well as the exit. And we obviously get the data, here at the entry of each function. so let me just run this and let's see how this works. let me just start this and where we start, client. So this is our PID and we run the tracer on this PID. So as you can see, it has attached the probes to a cell SL right? And now we have access to all the data that is being sent over this, TLS connection. So we have the client secret as well as the server secret. And you can verify by looking at, these secrets that are being locked here. And we have all of that here. So this is how basically you can use EBPF in your observability application. this is one of its application. You can use it to attach it to a lot more K probes. and go ahead. Go ahead and try and play with it and see. What you get. and this has been it for my talk. If you have any questions, you can hit me up on link and I look forward to hearing from you. Thank you.