Conf42 Golang 2021 - Online

Building Blockchains on the Go

Video size:

Abstract

We will learn about how to create your own blockchain distribution with GO, learn about the differences between a Database and a Decentralized Database, also explain when, why and how to implement a blockchain using a Microservices architecture with GRPC, and have fun creating or own genesis cube and streaming services.

Summary

  • We're going to be talking about building blockchains on the go using GrPC. Luis Cardoza Bird is a mobile developer and go developer. Also the co founder of this software technology company named Devotion. If you have any doubt of want to ask me something, please feel free to contact me in any way possible.
  • We're going to do a little example about how to create our own blockchain distribution using microservice with GrPC. Why are we choosing go as our main language to develop a blockchain distribution? And finally, we definitely aren't going to be doing a code session.
  • Blockchain is living aside all that complex theory that most of the blogs or web or website or even videos talk about. It has three layers. The data layer, the hash layer and the encryption layer. These three components are needed to build a single block.
  • Golan is a language that is easy to learn, if I can say the easiest language that we can learn. Golang is a static and strongly typed language. The go routines can be combined with any kind of other threads. The most used solution is going to be always the correct one.
  • The GrPC is a combination between HTTP two remote procedural call and also protocol buffer. The mission is to develop a blockchain distributor using GRTC and go. Here is a simple code example.
  • Now that we have our RPC, we need to generate that information. How are we going to be generating the information or the skeleton? Well, in my case I prefer to use a bash. So please be careful and please be advised that if you do any kind of change inside this file when you rerun the generate sh, it will delete everything.
  • We need to create a new blockchain. How are we going to be doing that? We need the previous hash and the data. But still there's something that is missing. We don't have a function to link those blocks rather quickly. Once we finish this project, you can basically copy chainer and paste it whatever you desire on a new project.
  • Main logic of the blockchain resides on this file name Chainer Go. We need to create a new structure to handle requests inside on a custom structure. There are missing methods to be implemented. To solve that issue it's just needed to add proto and automatically detect the changes.
  • Start streaming is basically a copy and paste. We have an implementation of a custom implementation using GRPC and a custom streaming service to fetch data. However, the information is being updated as you can see, and the hash is being retrieved by the client.

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Hello everyone and welcome to this session. My name is Luis Cardoza Bird. I'm a mobile developer and go developer. And also I'm the co founder of this software technology company named Devotion. In this meeting, we're going to be talking about building blockchains on the go using GrPC. Okay, so before everything, before everything else, we need to do some clarification related to why we are going to use go and also why we are going to be using GrPC. But first, I'm using to do a little presentation about myself. As I say previously, my name is Carlos Albert. This is my email. If you have any doubt of want to ask me something, please feel free to contact me in any way possible. So with that fulfilled, we can continue. Okay, so before we start, we need to think on some steps just to clarify everything. The first thing is, what is blockchain? Then we come to the second step. And that is, why are we choosing go from all the other programming language like node, JS, Rust, Ruby, C, C sharp, Python? Why are we choosing go as our main language to develop a blockchain distribution? The third step, what is GrPC? And finally, we definitely aren't going to be doing a code session. We aren't going to be doing an example. I'm just joking. We're going to do a little example about how to create our own blockchain distribution using microservice with GrPC. Okay, so with this we go to the first thing, how blockchain works. As many of you can imagine, blockchains you can imagine as a little cube. However, blockchain is living aside all that complex theory that most of the blogs or web or website or even videos talk about. We can simplify everything related to blockchain into this. Just imagine a simple block. This block. It has three layers. We have the data layer that is going to be storing any kind of information. We have another layer named the hash layer. And also we have another layer named the previous hash. These three components are needed to build a single block, just a block, not a blockchain. The term blockchain, as it say, by itself, is just multiple blocks that are nested one with another. So, continuing we have this example. We have our cube, its name, the Genesis block. Why Genesis block? Because it's going to be the root, the origin, the alpha, the beginning, the beginning of everything. So in this case, we have our Genesis block and it say, okay, we have data and is 1000. So the data on the genesis is going to be 1000. The hash, in this case, the previous hash, it doesn't have any hash. So in this case we can set up a hard code hash name and at the end the whole block we applies any kind of encryption. RSA, PBKDF, argon, two, any kind of encryption. It can work. So the third step, name encrypted, is just any type of encryption technique that you like the most. Okay, so with our Genesis block, we can build a second block. This second block, it has the same characteristic as the Genesis block. We have the data. In this case, we are storing blockchains of just integer numbers. In this case we have block 1001. This 1001 is the data. We have the previous data. That is, the previous hash in this case is 1000. Obviously, the previous value that we're going to be presenting in this case is going to be the encrypted value. But just for demonstrate how it works, I'm just displaying you the information unencrypted. And finally, the final layer is the hash layer. That in this case is just the encryption. Or if we can say it better, is the combination of the information on the data layer plus the information of the previous layer and also added the information of the encryption of the whole block. And with that, we already are building a blockchain and the same process apply for all the other blocks that are going to be building. So in this case, for example, block 1001, block 1002, if you can see the process, is the same. It's an iterative process, it isn't going to change. However, there is a special case. If you can see here block 1000, the data is needed to build the second block. This is the block 1001. And the block 1001 is also needed to build the next block. That is the block 1002. But there is a special case. What is that? For example, if we decide to move, or for example, if we for any reason move one block from the third position to the second one, the blockchain by itself isn't going to be matching the data. And that's the special thing about blockchains, the security that brings by itself. Because when a hacker try to access to the information, the hacker needs to decrypt all the blocks that the system has. But in this case, he will never going to be. Or, well, if don't say better. It's going to be really difficult if we say it may be impossible to retrieve all those blocks and also to retrieve the Genesis block. That this block is a special block because it has all the necessary information to start building the next generation of blocks. So as you can see, if for some reason we have block 1002. Yes, it matched with the previous data. However, with the next block it say, hey, wait a minute, this is bad because this block, the term one block 1002 can do any kind of matching with the previous block. So the block 1002 is bad, it's wrong. So that is the main reason why blockchain is currently being adopted by more companies, by more software projects, because all those benefits that come by itself. However, there's just one little thing that we need to make clear. Blockchain, by itself it counts as a database. So if you like to implement blockchains with a database, it's kind of difficult because the structure of blockchain is just to store information. And obviously a database stores information, but in a more complex way. However, blockchain, it can be used to more simplified data, for example, user credentials or user information, those kind of things. And that's the main benefit. Okay, moving on. We have the compiler about, well, Golan in this case. Why are we choosing Golan? The first thing is because Golan, the learning curve is really, really low, is a language that is easy to learn, if I can say the easiest language that we can learn. And also it has the big benefits of the compiler. This compiler, it works like you code once and it's going to run everywhere on every platform and every architecture, Windows 32 bits, 64 bits, Linux 32, 64, macOS, any kind of creating system. The final thing is, and also Golang is a static and strongly typed language. There isn't going to be more than one solution. However, the most used solution is going to be always the correct one. So go by itself, it brings you a lot of benefits. The compiler, the easy code, and also the static and strongly type language. And also, finally, we have the best thing of go, and that is the go routines that is basically using native threads. However can be combined with any kind of other threads, native trap and well, it can notify by itself to any kind of truth brothers or even the main threat. So these are some of all the benefits that go give you by itself. The next thing is GrPC. First of all, GrPC is a framework. We can resume everything on four things. Those are going to be the first thing is it has a high quality HTTP connection. But what I say about high quality HTTP is most of HTTP connection. As you know, maybe on some of your projects, personal project, or on some business software development. For any of you that have developed APIs, you know that by itself, the API, all the connection that has been done is using HTTP one. However, your pc, inject a little upgrade into that connection method and upgrade it to the HTTP two, that it brings you a better performance and also whole more clear results from end to end. The second one is framework, really easy to understand. We are going to see it on the example. The third one, the change that we are going to be doing on GrPC are easy to change because GrPC by itself is going to be segmentating everything on minimally service and those services can be rearranged, restructured without us to be changing or without us to be doing a lot of change on the server or on the client apps. And finally we have the assist that GrPC brings us that is most of testing, inspection and modification thanks to the protocol buffer that is being used by default on your pc. This is the kind of thing that maybe you are used to. For example rest API as I said previously, is based on HTTP, one that is added with reserves and that services. We are going to be using JSOn to fetch and send information. However your pc, most of the developers think that it's related to Google, but no it isn't. Well, the g change on each upgrade, sometimes it's good, great, genius, but never Google. And the GrPC is based on a remote procedural call. The GrPC is a combination between HTTP two remote procedural call and also protocol buffer. Actually is kind of easy. So with this presentation done, we can do a little code example. In this case we're going to be using go and the mission is going to be develop a blockchain distributor using GRTC and go. Okay, so now that we are on our favorite. Well in this case I'm going to be using Golan just because I really like how it handles all the go issues. But you can use any other text editor vs. Code or atom or, well, anything else. So now that we are using go, now that we already create our project, we're going to be doing some setups. First thing we are going to create directory name Proto. This proto is going to be used to handle our protocol buffer file. This case the skeleton for our blockchain. So we're going to type what name can we keep it? Okay, let's go with blockchain proto. Okay, that's cool. So now that we have our blockchain proto, we can do some setup. This case we're going to type syntax equals proto tree. Then we need to specify the package. In this case the package is going to be proto. Don't forget to put the dot and the comma, the semicolon. Then we're going to type the message. This message is basically your model class. If you come from Java or C sharp or any other language is our model. So the model that we're going to be using is a block model. This case we're going to type message block. We're going to open and we're going to type string. We need the previous hash. This preview hash is going to be the first data type. Then we need the data that we are going to be storing. This data is the information. So in this case it can be a string, it can be a number or it can be a boolean or even so it can be another object. But that depends on the need that you require at that moment. But this can be anything. Do you want to use a json? Just put a json. You want to use integer, a string or an object, go ahead, replace it. You don't have to do nothing more. Okay, so continuing we are going to add the final value that is going to be string hash. This hash is going to be the encrypted value of the block. So we have our block. The next using is we're going to be adding a block request. So we're going to add message block request. And in this case just string data equals one. Next message we have a request, we're going to send a response. Well, this request is just for the client. Basically the client send string as JSON for example, type API as example. And the value that we're going to be returning, or if I can say it another way, or your pc server is going to be returning is just a simple string and that is going to be the blockchain unencrypted. So block response, string hash equals one. But we are going to be doing something a little bit more complex. We're going to return a creating of blocks. It's going to be constantly fetching the data and it's going to be constantly sending the new information to the client. Going to type message chain string response. And this chain string response is going to be returning an object. What is going to be this object? Well, it's going to return the block object. So block is going to be inside the variable block and it's going to be the first type inside our model change turn response. Then we're going to type empty model. And that empty model is just using to be used when the user requests this. For example, a get that. As you know, the get it doesn't need JSON body to fulfill the message. So GRPC, in this case we're going to be using an empty message just to simulate that get petition. So we're going to type chain request. We're going to open and close. Then we're going to add message chain response and we're going to send a repeated block. Type block equals one. No, I'm using to type blocks just to be more sadly on the information. So what do we have at the moment? We have a message named block that is going to contain the information of our new block that we are going to be creating and also the other blocks that are going to be chained to the previous one. We have a block request. This is just for a singular block, a block response for another singular block, a chain string response that is going to be basically nested link list of blocks, chain request and chain response. This repeated is basically just a while. It's an infinite loop. Finally, we're going to add service blockchain. And in this service blockchain this is just going to be the functions as you know, on your controller. On your controller you can just type this, for example, get data and then return JSON. Just an example. This scenario is going to be replicated, but inside our protocol buffer. How are we going to be doing this? Well, in this case, the only thing that we need is just to add an RPC, a remote procedural callback, and it's going to be named add block. What is going to be receiving? Well, it's going to be receiving a block request. Then this block request is going to return a block response. That's it. We're going to copy this, paste it and stream get blockchain request. And here we're going to return a stream chain stream response and finally get chain same as before, chain request and it's going to be returning chain response. So this second RTC and the third one, they're the same. In one we're going to be returning a streaming, and on the second one we're going to be returning a repeated object, but this repeated object, it can be treated as an internal streaming service. So this one and this one are almost the same, maybe on a 90%. So now that we have our RPC, we need to generate that information. How are we going to be generating the information or the skeleton? Well, in my case I prefer to use a bash. So I'm going to create one. It's going to be named generate Sham. Remember mbash? And this is the comma to generate the skeleton we need to type proto blockchain, proto go out because we are going to be returning go code and plugins equals GrPC and a little dot. And with that can just run this command, CD, auto, sage, generate sh. Okay. And as you can see, GrPC add to the code, project a new file named blockchain. PB, go. Well, protocol buffer, go. This protocol buffer is auto is a code generated. So please be careful and please be advised that if you do any kind of change inside this file when you rerun the generate sh, it's going to delete everything and it's going to add the new information. So in this file you don't need to modify it. It's good as it is. Okay, so now that we have our blockchain pv, we can start coding. Finally, what do we need? We need the model. In this case, the model. I'm going to create a new directory named chainer. In this chainer we're going to be adding a new go file. It's going to be naming chainer. Okay. And we are going to add type. This type is going to be a block. But this block, if we can check, this is our model. We have three strings, name, previous hash, data and hash. And with that we can add previous hash. There is going to be a string add data also a string. And also hash is a string. And of course we need to be returning a list of blockchains because as the name say by itself, it's a nested list of blocks, objects. And with that we can just add a new structure name, blockchain. And in this blockchain we can just add block that is going to be an array lock. And we're done. Sorry, blocks and we are done. We have our block structure. As I said previously, this data and this data type. In this case is a string. It can be replaced by any kind of data type that you desire. And string a number, Boolean, Json, a list, an object depends of what you need at the moment. Okay. With all that we can start doing some more changes. What do we need? We need to create a new function. In this case we need to create a new blockchain. But how are we going to be doing that? Okay, let's create a function name make block. This make block. What do we need? We need the previous hash. We need the data. And both are going to be a string. And finally this is going to be returning an already existing object of type block. Finally, we are just going to be adding the block. Block is going to be an already existing block. And I don't why the autocomplete. Okay, there we go. Okay, so previous hash is going to be previous hash data. It's going to be data hash. We don't have the hash. Instead we need to decipher the hash and then add it. Well, with that case, we need to add a new function. We can call it get hash. Get hash because what is the value? This is going to be the previous hash on the data that we need to encrypt. Also, both are going to be string and it's going to return a string. Well, hash is going to be. We're going to do some encryption. In this case, I'm going to be using the sha two five six. Feel free to use whatever encryption method you like the most. In this case, I'm going to be using Sha two five six dot sum 56. And we're going to be adding an array of byte. And that array of byte is going to be composed of the previous hash plus the new data. And at the end we are just going to be returning the encoded value of hash. However, it's going to be linked or it's going to be differences with this little symbol. So in this case, encode two string. And with that we have our hash function. So we can just add get hash. And what is going to be the data? Well, it's going to be previous hash plus data and return block. And we are done. We have our block. We have our make block. But still I feel that there is something that is missing and that is we need origins cube. Okay, let's create it. Funk make blockchain. It's going to be returning an Albert existing blockchain. And we are going to create a new variable named Genesis block. And this Genesis block is going to be make block. As I said previously. Remember Genesis block. It can be any kind of information. It's just dummy day. Well, it's not dummy data in this case it's going to be. But you can type with any kind of special key, special characters that you desire. So in this case I'm using to type Genesis hash for the data also Genesis hash. And finally, with that we can just do this chain equals an array of type block. And this type block is using to be related to Genesis block. Finally, we are going to return a new blockchain. We're going to return a blockchain and this blockchain is just going to be nested with the chain. And there we go. We already have all the information that we need. So we have make block, get hash and also the creation of the whole cube. We are good to go. But still there's something that is missing. I believe that more than one found the issue and that is that we already have the creation of the block and the Genesis queue. Well, the Genesis block, but we don't have a function to link those blocks rather quickly. In fact, once we finish this project, you can basically copy chainer and paste it whatever you desire on a new project and just do some little changes just to be matching with your new code base and you're good to go. Well, we're going to be creating a function name apent block. This apen block is going to be needed inside our blockchain. So we're going to be adding a superpondition and also we're going to be requesting the data. There is going to be an string data type and of course we're going to be returning a block. Okay, so previous block is a variable, it's going to be chain blocks. And we're just going to be getting the whole size of the array, but we're going to be reducing that size by one. So length of chain blocks minus one. Then with that we can create the new block. New block. It's going to be make block. This make block is going to contain the previous block hash and the data, the new data that we are going to be creating. And finally we're going to be adding that result to our currently existing blockchain. So chain block equals apen just to add that new value. Chain block is the original array. We're going to be adding the new block. Sorry, my bad. I almost forgot to turn new block. And that's it. Okay, now we have everything done on our chainer. Now we need to be creating our server for the server. It's going to be rather quickly because the main logic of the blockchain resides on this file name Chainer Go. Well, we're going to create a new directory name services and in this server we are just going to be adding a new go file named server go. Okay, I'm just going to replace server with main just to be able to be capable of run this project. And we're going to be creating some needed structure just for the GRPC service to start running. We're going to create our funk main. And in the function main we're going to be adding a variable that is a listener. And this listener is going to be net listen. I'm going to be requesting a TCP connection. What is going to be the address? Let's just say 8004. However, this is giving us an error and it's because net listen throws more than one value also throws an error and with that we can just add another value name error. Finally, let's just going to check if error is different from nil love fatal. What can we type here? Let's just type unable to listen on port variable. It's going to be error perfect. Then we're going to create server. It's going to be a GRPC new server and server is going to be a new, sorry, server is going to be a new server. However, this server is going to be linked with the chainer. This chainer is just to start with our genesis cube. So it's going to be chainer make blockchain. This is giving us an error but it is because we haven't import all the required elements. Let's just add it using to be blockchain, the Gopher chainer. Let's just copy this paste and it's going to be proto perfect. Okay, that's good. Okay, now what is the issue that we have here with server services? We need to create a new structure. This is because we need to handle this request inside on a custom structure. In this case it's going to be type server and this type server is going to be chainer. Blockchain and we're good. Finally we're just using to register our new server or new service on the protocol buffer. So proto register blockchain server and this register is going to be chainer. Sorry, my bad. It's going to be our server and the new value server. But still we have a little issue. Say cannot use server as type blockchain server type does not implement. Okay. Just because there are missing methods to be implemented. Okay, let's just add it. And finally server serf listener on our desired port. Let's just going to add those imports context generate request. Okay. Okay, we're good. Now I'm, what is the, what is the other thing that we are missing at the moment? We have our server. We have a block stream block getchain but we haven't implemented that. Okay well if for some reason you are receiving this kind of error that say unresolved type. Well to solve that issue it's just needed to add proto and automatically detect the changes. So you're just going to copy this paste it, paste, paste, paste and paste and all the changes are applied as it should be. Okay, so add block. What are we using to be doing on this function? Add block is using to be basically block s chain because we're going to be appending a new block, append block and we're going to just send the new information that is going to be stored and with that we can just return a new proto and this proto is going to be block response and the hash is going to be block hash. And finally we just return also we just link anil because there isn't any kind of error. So we're good to go. Second one stream get block. How are we going to be doing this? Well, basically the stream get block is just going to be sending all the information that is on the list of blockchain. So with that we can just do this for anything in the range of s chain blocks. We are just going to be adding the current object into a new variable named result. It's going to be a new proto chain string response. And this changett response is using to have a block. This block is going to be of type proto block and we're just going to be adding this information, previous hash going to be block, previous hash data block data hash block. Well we can add the same data so we're good to go. And with that, and with that we can just say the services is going to be sending the result but we're going to be sending this result each second. So we're going to have a time sleep just to reflect the changes on, on a better, on a better perspective. And in the end when everything is done it's just going to be returning a neil. Okay, cool. And we have our server implemented. If we run it, let's say go run server server go. It isn't going to be doing anything but the server is already listening for changes. In this case we're just going to be adding some client just to be fetching information and sending the new information. Well, let's do it. Just going to add client. It's going to be a new go file name client package name because we need to be running this. And on the client, first thing we need to declare a variable named client that is going to be of type proto blockchain. Okay, so with the proto you can just do this. Just going to be copying this, paste it because we're going to be using a lot of information that we really have and we can start function main is going to function main. Okay, so for this scenario we're going to declare two arguments, name start and stream. Start is just going to be adding new block to the chain and stream is going to be receiving the, receiving the existing block. So start equals flag boolean name start false start receiving. Sorry, my bad. Start sending sending concurrent block to the server. Cool. Second one is going to be name string. It's going to be flat o stream, false receive streaming of blockchain. Perfect. And finally flag parse just to get the information. Okay, so what is going to be the first thing that we need to do? We need to create a connection, a live connection. We're just going to create a variable name connection. There is going to be a new GrPC dial and this dial is going to request the target. The target in this case is going to be all address. In this case was a four going to copy this and paste it. And finally we need to specify how it's going to be taking the connection procedure. So it's going to be insecure connection. However, it's giving us an error and in this case as before it's because we are assigning the response but we are receiving the error. We're just going to be adding the error. And then if error is not nil then we're just going to let fail f cannot dial server and we're just going to send the error. Then client is going to be proto new blockchain client and we're just going to be sending the connection. Okay, finally if star is true, then we're just going to create a new function. It's going to be named start block blockchain. If stream is true, then start stream streaming. Okay, finally we need to create those two functions. Start blockchain. Sorry, my bad. Start blockchain. And we're just using to create an infinite loop. In this case we're just going to do an infinite four, in which case the information is going to be block. It's going to be client add block. We need to send the context, context background. And also we are just going to be adding the request. It's going to be proto block request this block request. We are just going to be adding the data. In this case the data is going to be the current time, so time now string. However, we are receiving an error. So same as before, we need to add another variable, but in this case going to be address error. Finally, if address error equals new sorry, it's not new, then we're just going to copy this using to paste it and we're just going to say unable to add block address error. Cool. And then if we don't have any kind of error then we can just print a new message name new block hash. It's using to be kick assigns the string so we can do this instead. Block hash. Okay, cool. And finally time flip because we're going to be adding a new block each second time second. Okay, cool, perfect. This case we're going to start running our client. Oh my bad, I forgot to do the change on the client this case and use variable client client. I'm just going to remove this and start the services. Go run client, client, go. Where was the argument? Start string. The purpose. So as you, as you can see, this server is already sending the information and it's returning the new encrypted hash. So it's working. We're going to let this blockchain run and we're going to finish the streaming. So I'm just going to redo all this and we're just going to be adding the final function, name function start streaming. Okay, so with the start streaming it's the same as the start block. Well, it's almost basically a copy and paste. So we can just do request equals a new proto chain request chain request. Good. Then response string because we're going to be fetching information. It's going to be client string, get blocks context background and we just be fetching a new request. For any scenario we have an error, then we're just going to be type a new fatal exception error while calling stream and then we are just going to be iterating over that response. So for block stream we're going to create a new variable that is going to be receiving the value of the string. So resource string equals sorry, recover value. And if for some reason the error equals to end of file, it means that we finish fetching all the information, all the blocks, so we can say log. No, we're just going to do a break here and we are going to just type, we have reached the end of this string. Finally we're just going to be checking if for some scenario we have an error that is different from Neil, then we can just lag fatal f, say error while reading stream. And after all this we can just create a new variable that is going to be fetching the result block stream, sorry, block stream with n, not with n, blockstream block and then block print format, in which case we can just type previous hash. It's going to be a string, then data, it's going to be a string, then the hash, which also is going to be a string and just using to be adding block produce hash, block hash and block data. Sorry, my bad. Block data and block hash. And that's it. We have the implementation to fetch all the blockchain. So let's just run it. Go, run client, client, go and stream through. And as you can see the information is being fetched by the client API, and as you can see, the information is being updated, the hash is being retrieved. However, the previous data that was encrypted is already been displayed on the console. And as you can see, we have an implementation of a custom blockchain distribution using GRPC and implemented a custom streaming service to fetch the data and also to send the data from client to server and receive the data from server to client. And with that we conclude with this speaker. I hope that it's going to be useful for you. And again I say thank you for being on this creating it means a lot. Don't forget, keep coding, keep practicing, and always go for the best. See.
...

Luis Cardoza Bird

Founder @ DEVotion

Luis Cardoza Bird's LinkedIn account Luis Cardoza Bird's twitter account



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways