What if you could work with some of the world's most innovative companies, all from the comfort of a remote workplace? Andela has matched thousands of technologists across the globe to their next career adventure. We're empowering new talent worldwide, from Sao Paulo to Egypt and Lagos to Warsaw. Now the future of work is yours to create. Anytime, anywhere. The world is at your fingertips. This is Andela. So welcome to today's talk in Peernet. My name is Akilan Selvacoumar and I'm a software engineer at Peernet and I work towards the Peernet core repository. And today I'm excited to talk about Peernet and the core protocol we've been working on, or the new standard and implementation of our peer to peer network, which we built purely using Golang. So just to recap or just a good way to start or a high overview of Purenet is Purenet is a peer to peer protocol designed for file sharing. But what we have done is we have actually used the best techniques available out there for sharing files between two peers, learning about peers in the network. We have implemented our technique, or the best techniques used for escaping that. And because we'll talk about the whole protocol is purely in pure go. So the advantage of this is we can run completely cross platform. So if you run on an Android phone, you can basically run pure. Net natively on an Android phone and various other architectures like Risk V and so forth. This is the reason why we've written purely in go. And also another benefit is it's easy to extend your application on top of Peernet because it's a single binary. We have a good set of APIs, and if you're a go developer, you can actually call function calls from the packages to make sure it's something customized to a certain function you're writing in your go application. But in a higher overview, Peernet is a simple peer to pay protocol for sharing files, and we're building a browser on top of it, which helps in file sharing to regular users. But for now it's only a Windows machine. But today's talk is just about the Purenet standard and the implementation we've done so far with our open source protocol, which is available on GitHub. So just to start, these are the design considerations of pure. Net. So the first packet must establish a connection we're talking about here about we do handshaking, which does this. We want to secure against eavesdropping. We want to ensure that there's no sniffing or spoofing. The third point is we want to secure against message forgery. So we want to ensure that the message is coming from an actual peer. For example, peer B gets a message from peer a, it is from peer a and not from some other node called peer c acting as peer a. We want to secure against tampering can extension, and by extension a bitflip. Basically when we transfer packets maybe over a larger network or two nodes between completely different countries, there's a chance that the packet could get corrupted due to some reason, and we want to ensure that we can identify this. Either we can identify this and correct it, or just retransmit, or ask for a new packet which is not corrupted. The fifth point is we want the packets to be as small as possible. So if nodes are far away it increases deliverability and lesser packet loss. And the last point is we want TCP like acknowledgments because Purenet is built purely using UDP. We want UDP to act like TCP. So we basically want a reliable version of UDP and we use something called UDT, which we'll talk about later. So this is Purenet architecture at a high overview. So what you see on your right side right here is a browser command line and the Peernet core which is the implementation of the standard slash protocol which is in Purego as you mentioned. So basically the entire core, or the entire core library comes built in with the command line project itself. So it's part of the command line package when you build the binary and the browser interacts with the command line using a set of rest API calls. If you could replace this browser with your own application as well on top of it and do a set of rest API calls with Peernet running locally on your phone and you have a peer to peer network overlay. But this is just Peernet at a very high overview. We're going to talk about in the next slides. We're going to talk about few of the important modules independently. And just to recap, we do UDP connection with other peers in the network as well. So let's talk about blockchain, our fancy link list. Or I like to think of blockchain as a fancy link list. So the blockchain does two things. The first thing we use it to store file metadata. The next thing we use it to store user profile information. So what happens is when we have a blockchain, we share it across other peers in the network, and then once the other peers get the blockchain, they know about a file metadata. So when searching they can just look up our blockchain and see if we have the following file. If we have it, get that information from, get that information from up here or download the file. And something interesting is our blockchain is actually kind of mutable. So that means the blockchain does not always grow in size. So if we delete a file, that entry also gets deleted from the blockchain. So how does this work? Doesn't it define? So how does this work? So basically what happens is if we delete a file rather than removing that entry from the block, what we do is we remove the entry and at the same time we change the version number of the blockchain with a new version number and then we propagate that new version number. So within the minute all the nodes in the peer to peer network should have the new blockchain version number or with the new blockchain itself. So what happens in the end is Peernet is kind of like a Merkle forest if we want to think of it with a ton of blockchains, which can be like a Merkel tree. But basically every peer in Peernet has their own blockchain and they share it with each other's peers. So all the peers will have a bunch of blockchains of other peers in the network. This will help using during search specifically. So let's talk about the packet structure of Peernet. So in Peernet we have something called announce, which is a pseudo generated random number. And basically this helps against preventing replay attacks and so forth. The next one we have is something called a protocol version. By default it's zero for in our case because peer nets in the initial stages, but eventually we'll have one version one version two. And this is just to ensure that two nodes can communicate with each other. This is just to ensure two nodes of the same version communicate with each other. Or if we have something like backwards compatibility, we can just address it by ensuring all nodes that within the packet itself that both nodes are of the same version. This is obviously a future using, but not right now. The next one is command. We'll talk about this in the next step. The next one is sequence. We're talking about packet sequences over here, pretty self explanatory. The next one is the size of the payload data, basically the size of payload you're sending. And the payload over here could be anything. It could be a part file, it could be maybe a get block message, something related to the announcement, depending on what the user is sending through the packet. And the last one is the randomly generated packets. Now this is kind of important because this is to ensure that a router cannot just say this is a peer net packet based on the size of a packets. And basically this protects against fingerprinting. So yeah, this is one way we use to skip the firewall. So part of our packet itself, the last one is in pure net, most of the pack. For example, if you do get block or most of the commands, they are encrypted using salsa 20. But if you do a file transfer, for now it's actually unencrypted and it's rather in plain text. So the reason for this is because of efficiency reasons, there has been like high cpu usage when we encrypt and send for each and every single, when we do a file transfer, because in file transfers we send a bunch of packets, because of course it's a file, we're transferring a huge file to the network. So we generate multiple smaller packets and transmit them through. And of course signing and sending actually slows down, increases cpu performance. So this is an open problem. We're still looking into it. We want to find an efficient way to anchor packets and send them over for file transfers. Next one is peer discovery, something simple. So one way is we use something called bootstrapping. So initially when you start Peernet, you would connect to a set of root peers. These root peers are basically Peernet servers, basically running Peernet itself. And basically it's a single point or it could be multiple root peers where all peers connect to these root peers. To learn about more nodes in the network. This helps immediately learn have a larger peer list, and this means we can find more files in the network increases, dispersing more peers, increasing our discovery to other peers. This looks like a centralized approach, but this is the only time. Maybe it's centralized, but you could have your own set of root peers or your own set of bootstrap nodes, and this could be even some computer in your house. So when you're running a private network, you want to ensure you learn about nodes in your private network through your bootstrap node or just through your root node so you can have your own list of it. But for now we've provided punit nodes to ensure you can learn them quickly. The next one is exchanging pls, basically exchanging, this is just part of the bootstrap process, or exchanging pls is I communicate to node b, I share my pls to node b, node B shares his or her pls to node to node a, which is me. And then I learn about new nodes from, let's say node a and node b knows about new nodes from me and so forth helps about learning about new peers in the network. The next step is if we want to quickly discover nodes in our local network, we use two things. One is an IPV four broadcast, and the other one is something called an IPV six multicast. So with an IPV four broadcast, we send, let's say an announcement message to a single IP address to actually a broadcast IP address in the network, and then that's sent to other nodes. Basically that broadcast IP sends the announcement messages to all nodes in the network. And whichever node is running Peernet in your local network will respond back to the broadcast address itself or respond to our node actually, and then we would know about that new node or new nodes in our network. And IPV six multicast is sending a datagram to a specific group of nodes in the network. These are just two techniques we use to learn about more nodes in our network. So we use two techniques, escape NAT. One is something, one is called UPnP and UDP holepunching. So UPnP just stands for universal plug and play. This means this is actually only certain routers support UPnP. This depends on your router model and whether it's enabled or not. And what UPnP does is something really simple. You tell your router, open the following external port for me and map it to my computer, which is running Peernet. And this will ensure that nodes outside your network can communicate through you by escaping NAT. And of course this is only if your router is enabled. And many people also disable it, sometimes for security reasons. So let's say you don't have UPNP. Another way you can do or PNN automatically will do it for you is using UDP hole punching where you'd use another server or actually use another node, which is any node in the network which is escape NAT to communicate. So this helps if two nodes are behind nats to communicate with each other. We use the rendezvous server, which is our middle server, and for example client a. Basically what we do here is both nodes, both of us connect to the middle server to know about each other. And basically it's useful if both nodes are behind the firewall. So for example, what will happen is client a will connect to this. Let's say this node behind Nat and this node behind Nat would communicate to client b stating that the messages come from client a and then client b based on that information about client a from the rendezvous server or their middle server would directly do a peer to peer connection with client a. It's a widely based concept, escape Nat. You can see it commonly used a lot with WebRTC as well. And we have it within ourselves because our protocol is UDP, it works perfectly. We can easily implement holepunching and we already have it implemented, tested and it works really well for nodes behind nets. The next step is, the next one we want to talk about is something interesting we've been working on. It's something really simple and we intend to improve a lot more on this later. And this is called a decentralized search. So decentralized search is really simple implementation. So in a peer to peer protocol we don't like searching using hashes or maybe a file name or basically some IP address with the file name. We just want to search for a keyword and then we want all the results of nodes in the network with that keyword. So what we do is, I'm sorry about that. So what we do is the nodes with these key. So basically when we search. So let's actually step back a bit. So when we upload a file to the network, what we do is we generate something called indexes. So basically what happens is we take a file name and if there are spaces in the file, we consider them as separate words and each of these words are indexes. For example, we can have a file saying go first, talk 2022 mp4 and each of them with a dash or space or an underscore. We remove those underscores and each of those words are an index. And all this index is mapped to the hash of the file. And basically if we search using this hash of the files, we can get the file metadata. And using this metadata we can either download the file or even get more information about the file from the peer to peer network. And that's how we do indexing. And that's why we do indexing. And when we search, we also do the same thing. When you search, let's say, for something specific, we make sure that all of them are lowercase. We remove spaces, we remove quotations, and then based on your search, we search against our index table. We get the multiple hashes of the files and then let's say we query them, let's say from our blockchains we have, and then we get the metadata and we display it. And then that's as simple as that. When you search a file, how you get the results of your files in the network. Actually we'll do a live demo which will make so much more sense of what I'm talking about right now. So we have, we'll just jump quickly. So this is something also super interesting. And this is a warehouse. This is where we actually store our files in Purenet, kind of like our file system where we store files. And let's talk about how a warehouse works. So as we know, just to recap, a blockchain only stores metadata, whereas a warehouse actually stores the files, as I mentioned. So what are the features of warehouse? What we have is automatic deblication. So to ensure that it's kind of like when we create our little Merkel tree for every file, we do not have redundant data. We address files based on hash. This is important content based addressing. So you search a file rather than the files name, you search based on the content or based on the hash. We can do three simple actions from our warehouse. We can read a file, write a file, or remove a file. The next part is something interesting. We can read parts or chunks, or we can seek parts of the file. So for example, if a peer is downloading a file from, let's say node b or peer b, basically peer B can also seek a certain part of the file and transmit it over to node to peer a, which is important. And we can store, and the next part is we can store file as large as what our disk supports, which is trivial and obviously should be the case. This is something, and this is, I would say the last part about the technical bits of Peernet. So this is something we've been working on for a long time. It took us a long time to integrate. So we can elaborate about this. And this is something we are very happy to announce that we have full support and we've fully written in Golang about this. And this is the UDP protocol. UDT protocol is basically making UDP more reliable, making UDP act like TCP, but except most of the transport layer or most of the implementation like congestion control, is on the application layer. So the benefit of this is that, for example, when we know two nodes are close by to each other, we can actually have a larger packet size because we know there's going to be lesser packet loss and this would increase. Essential. We're utilizing most of the bandwidth. We can modify ascending window size, we can modify a lot of things which normally can only be done at a kernel level. And because we have all these major things at an application level, like sending acknowledgments, receiving, sending a NAC packet not acknowledged packet, having our bucket of peers, having a packet of packets not sent, setting the size of the packet, maybe even defining how the packet looks like. Like how we mentioned in a previous slide, having all this at an application layer ensures that it's cross platform. We can really optimize it to utilize most amount of bandwidth available and the stack. And yeah, just having these two benefits makes a big deal for us and over the future is going to improve the protocol dramatically with whatever we have plans with. And Unity has won multiple bandwidth competitions and making this work in Purego was not easy. I think we're the only project that actually fully utilizes UDT in pure go. So we're glad about this and we want to improve UDP because we fully have a lot of things to improve on. But this is majorly the backbone behind how we do file transfers between from the most lowest level but having it completely implemented in Golang at an application layer. So this means that you as a contributor, it's going to be really simple to modify it customly for your use case if you want to get to the finer details. And you wouldn't have to change anything on your operating system or kernel itself. So let's jump to a lifetime of Peernet. Finally we're here. I'm just going to quickly open my terminal right here. So let me get a clear command. So what you see here is the command line repository. I will, sorry, 1 minute. Quickly, right here. Quickly put it right here. So this is the command line repository as you can see here. All you have to do is clone this repository. I will show you the link soon and all you do is go build and that's it. It takes a few seconds, obviously I've built it so it's all in my cache, but normally it just takes a few seconds to build. And once you're done building you can just run that binary and you have right here. Let me just position this in a way so that you can see what I'm talking about. Okay cool. 1 second. Okay here so we have a set of commands which we have improved. I'm just going to show a few commands right now. We just wanted to show that this works. So if I do status, sorry. I can see a set of nodes in my network right here. These are basically public IP addresses of all nodes in a network. So we identify each node by their public IP address. We can get more information. If you want to find more information about each node we can do something called pure, pure bliss which is right here. We can see all the adapters available in these specific nodes, which could be, which is substantially useful for us. But for now, this question mark right here, we have these other commands. We're not going to talk about this. Today's talk, you can play around and easily figure out what they do, since they're pretty well documented for what they do. But just explain, I'll explain a few important commands. For example, if you want to test out and see how your file transfers, you can use something called probe file transfer. So what this does is you can have a file added to your warehouse in node a, and node b also has that file added to the warehouse and you can just do probe file transfer. You put that node id of node b, for example, of who you want to test against, and you put your file hash and this will basically do a file transfer and show you the speeds you're getting and ensure that the file is fully transferred by comparing the hashes towards the end. And this is a great way to debug and ensure that the changes you're doing are working, how efficiently they're working, and so forth. So just wanted to mention about the command. You can also of course try out the other commands which are super interesting. As you can see, we're going to add commands soon in the upcoming releases where you can actually just add a files, remove a file and do the basic stuff. But for now you would have to use the browser to do that. Or you could read the rest API docs which help you add remove files. But we will soon add them to the command line which is using certain functions which will make it easier for us to integrate directly to the command line without having any complicated code. So just bear with us. We will get the basic functionality like add remove files soon so that you can directly talk with your friends peer from any creating system just using a few command line instructions. The next thing I wanted to talk about was actually the peony browser itself which is right over here. You can see this is proprietary, but this browser uses the punt protocol itself. So you can see right here, if you go to the explore tab, you can see a set of videos recently uploaded to Peernet itself. If we go to the home tab right here, for example, I can type Ukraine if I'm spelling it correctly, and just give it some time. We have a set of just this is to prove that our peer net search or search works in our peer to peer protocol as per how we explained it in the slides. You can go here. If you hover over your peer, you can find out from which country it's been uploaded from. If you click play, you can play the video as well. Currently this is a bit slow, but this works fine. This is actually an older release, but if you do check our Twitter pages for the new releases with the new video players and something interesting over here is that when you play a video, the video is played on memory and not on disk. So yeah, it's only temporarily. So it's great. First, eventually it's going to be great for streaming videos as well and actually already is, but we haven't released that version, so stay tuned to get that version from us. So I'm going to quickly jump back to the slides right here. I'm going to go here view that was just a very quick demo of Peernet to understand what we've been working on and feel free to try it out yourself. We love to see the ideas you like us to implement next. So these are a few ideas on how you could extend Peernet. This is just our few ideas we found interesting, but of course you can implement it using your own ways. One way is to ensure that you build a distributed game engine in such a way that when you're playing a game or you are loading a map, you could basically load a map from the peer nearest to you by just searching based on the hash. IPFS has a few examples. It could be interesting to also extend it with Peernet because in Peernet over time we are going to be way more efficient in terms of discovering peers, transferring content between peers using our implementation compared to other peer to peer networks. But this is underbuilding and we're very soon to accomplish this. So one is building a decentralized game engine. It'll be very interesting to see how if you could have a decentralized game engine on top of Peernet, another one we really want to improve our NLP searches. Sorry, we really want to improve our searching. It'll be interesting if we could actually someway embed NLP to the Peernet protocol as an external open source project. And potentially we could also add it'll be interesting. We're also keen to find out how we could introduce files ranking when doing decentralized searches from, if possible, the core protocol itself to improve results. But these are a few ideas we think are interesting, but it's your creativity. We just want you to play around, try Peernet and use it for your use cases. So how do you contribute to Peernet? It's super simple. You can join our Discord channel or forums on our website. I'll show them the next slide. The links to discuss about Peernet you can do is you can also create a pr at our GitHub repository. If you want to contribute to changes to the Peernet protocol based on modifications you did on your fork or your project, so forth, it would be recommended that you create a pr and we discuss through the changes you want so that we don't have to reject the pr you create. So yeah, feel free to discuss on our Discord channel or just create an issue on GitHub and we'll get right back to you. The last way to contribute to Peernet is if you have a large proposal, say it's going to take you more than four months or even six months to complete. The best thing is to email us at info@peernet.org so that we could potentially personally talk with you and try to ensure that your idea, which you want to merge with Peernet, is towards all our interests and we are on track and we are all on the same page when discussing about it. So these are a few things. So just to summarize, pure net is a completely decentralized network with no intermediaries. We have a full working browser and a command line implemented. We can escape nats and firewalls to a certain extent. Our entire implementation is a single binary file and you need zero configuration to get it started and running, which is super cool. We control most of our transfers at an application layer, which means we can optimize our transfer algorithms to take advantage of bandwidth available. Our implementation can run cross platform and we don't have to really contribute to the kernel so everything can be done directly to Peernet. We can ensure also that we can also control packet sizes and have various controls, which is very useful when we do something like file transfers and so forth. These are our links to the Peernet project. Feel free to look at them. I'll just quickly go through them with you right now. So here we have the Peernet GitHub page, we have the peony core repository right away and you can see the Punet core. If we go back to Punet itself, we have the Peernet CMD also. And yeah, this is the best way to get started. Just clone the repository, build it and get started. And if you want to find more information about Peernet, you can also go to peernet.org and you can actually download the browser from here and get started right away with Peernet itself. And of course the last point is join our Discord channel. We'd love to discuss your ideas to use Peernet or your feedback or how you would like to use Peernet in the future. For your development. So thank you for tolerating me. Or thank you for listening to this talk today. And I hope to see you next time. Or we hope to meet you soon in our Discord channel. Thank you,