Hello everyone. I'm Andre and I'm here to present, the Rabbit Hole of Dependency in Goal and let's slowly get started. First of all, again, I am Andre. I am a principles of twin engineer at Delivery here. We are using Goal a lot in all of our services, and I have a lot of examples. Good and bad, how to work with, dependencies, and that's why today I want to show you, some problems we had in our code base. So if you want to, contact me and discuss some of the topics, you can easily, contact me by any of this. LinkedIn or, Twitter or Medium, whatever. Please feel free to reach me out there. yeah, let's slowly get started. And first of all, I want to ask you, can you clearly name couple of dependencies in your current goal and project and why do you need them? Just take your time, think a bit about this and, the next question I would like to ask you, can you clearly the name, couple of indirect dependencies in your goal and project, and why do you need them? Again, take maybe, press some, pause on the video and think about, but during the presentation, I would, ask you now to think about this. and yeah, but we are going through. first of all, I prepare a very simple goal program for you and yeah, you see it almost. It doesn't have any dependency. It doesn't have any glue in the program. It uses very default print end and it prints volt. So if we just take this and compile our Compile it file will be, or almost one and a half megabytes. this sounds decent, is a B. We in my presentation. Let's just to use some imports. Let's import very another very standard library from their goal. And it's FMT. Everyone is using it. It, I can't imagine any goal and projects where this is not being used. So we are using FMT to do the same. Just to print Hello Vault, and check how cha how the size of our application is going to change. Surprising, surprising. It's actually changed it, and it is two more, one more megabyte in the project. So if you compare, the size of our application, it's actually changed it almost in half. It's plus 50% of compared to the previous result. And, why this happens, I will explain a bit more in the, And let's look, if we add more dependencies into your code. So we added in o why we agent, because we want to read some, we provide in when we on the program, and maybe I put them, for example, just put my name. Hello. and, if you go into compile this, the compile it, size of the competitive program almost doesn't change. And this is something which, already should, rings a bell. What is going on? Actually, I just edit maybe huge, another dependency to the code, but actually it's not reflected in the, application. Yeah. And here you can see, if you compare it was 51%. Right now it's 53. But yeah, not the big change, but the answer, to this, not, this is not a problem, but answer to this, puzzle, lies in the FMT library. If you check FMT Library, you will find out that, in print go and. Some other of them you actually already include in IO and o Libraries. So this is a hidden dependency coming this FMT, when you include it, all of this is actually used. for example, reflect io. We are using, I don't know, for output in your prints to some re writers. It's very common in GoLink reflect you might use if you, for example, when you format it, you want to output the type of your, variable you providing. Or if you want to print, some information about the content of the object, SST Con we're using to convert integer, or not only integer, but numbers and into strings sync is puzzle me why we need sync. But most probably, it has some usages in the packages. And yeah, for sure you have Unicode because you made play around with Unicode. So when you include fmt, you also include a bunch of other dependencies among this. This is something you need to understand when you are working this, your program. So yeah, this is what I already mentioned. Agent FMT, you bring a lot of hidden dependencies and this hidden dependencies, they may have another hidden dependencies. and we can go into this rabbit hole very deep. Maybe I don't recommend, but yeah, we can do this actually. So if we, try to do something more sophisticated or what I wanted to experiment, I, okay, if I move my coat to some external library, how this is going to affect the size of my package at the end. So I created external library. This is. Of this library and, yeah, I format it as a library. I upload to the GitHub and I imported it in my program. So it's a simple function, message, print, and when I compile it and when I check the size. The size almost didn't change. So moving your dependency to a external library doesn't, bring something new actually, because you already have all of this in your program. So yeah, if you compare size almost didn't change. Absolutely the same. So then the need to ask ourselves a couple of questions, what if we try to do something more interesting? And like we were playing around with pretty simple quote before we were just, printing some out. What if I need to, print out the list of, S3 buckets I have in my account? And the quote for this program will be also very simple, compared to the previous one. So we have some external dependencies we introducing, Some generic library for AWS. We have sessions because we need to authenticate. We have S3 itself, and then we create in session, we create, a new service. And the call please lead the packets and then go in, in the, in the loop to print all of these buckets. Very simple quote. What can go wrong? nothing can go wrong, but what will the result? Let's check what this is a result. So when we compile our application, actually the size of the application jumps a lot. And this is interesting, thing I would say because if you compare like our size of application, actually increase it by almost 900%. So these goers here exactly for completion to show how the size of the application increasing, so you really can see it jumped to a bigger number. So why this happened, if you ask ourselves, yeah, maybe this is a huge package. It has a lot of APIs, it has a lot of, different objects. So maybe all of this together creates a huge, dependency. Hidden dependency, injecting, and what I thought. Okay. AWS states, for web services, this is Amazon Web Services, so it means it has some API and we can call this API directly, and just get the same result. For sure. It has some, complications about authentication. We need to sign our request, but we can figure out this. So what I did, I just asked Chad GPT to generate the code. This code actually work and the code itself doesn't, that is not interesting for us at all. It works. You can believe. What is more interested for us as this list of, dependencies, we have, so we have a working court, which can call list of buckets. without all of these, AWS dependencies, this is clearly only from the, default, goal length packages. So when we compile it, our size is, has decreased. It's less, six megabytes less right now. but still it's eight megabytes. And the program I was showing before, it's actually, like hundred lines of code. It's not something huge. amount of, dependencies. We important is also not that huge. why do we have these, problem these. Still, I would say inflated size of the application. And yeah, you can see it decrease it a lot. It decrease it almost 400% compared to the previous one, but still it's big. What can we do to solve this problem with our application or why this happened? and again, we implemented only one EPA call. In our program, and this S3 has multiple calls, to list object, to add object, remove object update to get versions. It's huge amount of API calls. So maybe the six megabytes we just, removed from our application. The six megabytes is, exactly implementation of this code, but still. Our application is still big. It's still big. if we compare to the previous one, it was two mega pies. Right now it's eight. So these six more megas of the program. So which of this dependency take, all the. Credits for being, for abusing, the size of our application. So I did another experiment. I started to remove one by one. This crypto sounds a good candidate to remove. So what I did, I commented code to still keep, the program they were can and try to compile. I did it. so yeah, I remove it. Crypto, there is no crypto independencies anymore and I compile it. I compiled it, but size of application didn't change much. You can see the again, on the four kilobytes. okay. my thought was let's proceed removing more and let's find out which one actually is the reason for this inflated size of application. So we also remove it, Yeah, there is no encoding in, our application and run and. This is not the mistake, actually. this is two different screenshots, but unfortunately, there is no effect. Nothing has changed in our program. So most probably this accordion comes with another dependencies, as a hidden one. So yeah, we need to build the three of dependencies to understand, what could be abuser of our size. yeah, still we have eight megabytes. Nothing has changed that much, so I decided to proceed to removing stuff. So I remove it time library, and nothing has changed. Almost. I remove IO and the same, we know I already included in fmt. You not bring anything. But we have on the FMT Os, which is also included in the FMT as a hidden dependency. And we have net fm, net HT TP. And what I did next, I created a simple program using only, net htt p just to call, random URL. And yeah, if we compile it, actually it bring it. Our application becomes like five megabytes. So yeah. And net HT TP is another very common library we have in Golan. maybe 99% of application using it because we need to communicate our application with other applications. So we are using it so it brings another batch of dependencies and yeah. And why I'm bringing this, I am for sure not encourage you to remove all your dependencies from Sakoda base, and I'm not encourage you to remove or implement nett TP on your own to have zero dependency and very light code. And, but what I'm trying to say, most probably you would never, bother yourself to, Look into size of your application when you remove something or add something, and you didn't know that how many dependencies it brings when you add a small lever to your application. And yeah. And maybe net CDP is not the biggest one, but this is common one. And when you add it, you really don't understand what it brings to your application amongst this, what the dependencies of dependencies go to your application. And what I want to bring into the table that when you work with, in the beef company, for example, like delivery here or even smaller. You need some, culture of dependency management. If you don't have it, you can face some issues. And the second part of my talk will be, again, exactly about these issues. We face it in our quarter basis, couple of times. And, I mentioned in this slide, yeah, it's you risk to have very poor code quality. It doesn't really, go to these bad things, poor code quality, but it leads to a lot of technical depth, which you can remove forever. Okay, and proceed with my presentation. I want to give you some small example. So just imagine you have a service, some global service. I call it the STS because this is common name, around the world. And, it stays for, service to service tokens and this service commonly used in different, ecosystems, to create a token to, communicate between services. So let's imagine we have this global service. Since this is a global service and everyone need to adopt it, we also want to support our teams to not implement the well every time, but reuse, as something. So we create some library. So in different languages, Java, python, Ruby Rolling. We implementing some help levers, which abstract communication or abstract some. This a P layer for your application. So some teams just need to plug and play and use this library. sounds easy and sounds like a common sense. And this is what we're using in on daily basis all the time. So we have this service, we have this library, and we have, service a, this is just some business, service, which executes some business and it needs to use this, service to service token. So we use this library and yeah, what I forgot to mention, the library has version and version one. This is important, for future. And yeah, we're using everything works and then team decides to implement version two and what they actually do, unfortunately, and this happens, this happened in my career couple of times, and in Delivery Hero as well, guys, implementing new features and changing the contract. The change in interfaces. And this leads to inconsistency and low record compatibility between version one and version two, and what happens next? even if team, of service a, if they want to migrate to a new version, they would need to, do some extra effort to do this migration because interfaces change it. they need to rewrite some code. And maybe not in one place, but in multiple places. This leads to the case when team decides to stay with version one because it works for them. And in software engineering they have this proverb, if it works, don't touch it. So the team is not going to touch it. It'll stay and they'll not going to use version two. And here another problem because, like teams who create in, these global services who maintaining the libraries for these, global services, they also don't have a lot of capacity. So they can maintain both versions simultaneously and usually they say, okay, this is deprecated. We are not supporting it. We'll not implement new features. We'll not implement security updates. You all of you should use this new amazing, shiny version tool. Okay? So team of service A stays with this technical depth almost forever, I would say, and then happens what happened in my experience multiple times. We have another global service. And, it's a global service doing some stuff, but, and also another team decided, okay, to help team needs to implement this library, to support, easy. Plug and play, for other, for our consumers. Okay. They did, but they also use this, service to service token. And for sure, since they creating a new service, they're not going to use old version. They're going directly with version too. Which has different interfaces compared to the version one, and they starts to create a problem. If service a wants to use, this service and they use this library, it may have some dependency and, dependency on what is implemented in SDS. And in this case, when you try to install the library from the. another global service. It won't work. What it'll tell you. It'll tell you that, oh, I see, different implementation or of some functionality, which has the same name and it stays for the same, library. Actually, it'll fail. And this something what happened, in my company couple of times. And this is something, really huge problem for us. and if you look what the problems, yeah, it's hard to maintain multiple versions. As I mentioned, the team, which created, this library, they may not have capacity to maintain version one and version two and keep different interfaces. These are hard to push as a teams to migrate to new version. As I said, they also don't have capacity and for them to, assure their product people that technical debt is something bad and we need to migrate, it's usually very complicated. At some point, the loop will be closed and you will end up with some services are not able to use your libraries. As I mentioned in the, example, Above. It actually takes a lot of time to end up to the situation. This is a problem because what he described, it didn't happen in day or week. It happened during like year or couple of year. And then this situation evolve with evolve. But you may develop more code in service A where with this old dependency to, old library. And at some point it's. Super complicated to avoid this technical debt or get out of it. You have years of development of on top of this and it's more and more complicated to change or write your pot and it's really requires a huge investments to fix the problem. And sometimes people even say, okay, it's really easy to start a greenfield project rather than trying to solve it in place. and yeah, what is the biggest problem, was in this case I need to mention because also the team who is maintaining the library, they did the mistake. they need to, acknowledge this and understand, when they were, create a new version. What they did, they just said, okay, we have version two, and they changed it in version. So in the same version, it shouldn't be changes in the contract actually, but unfortunately the guys did this and, it created all of these, following, problems. What they had to do, and usually most probably working with core programs, you notice this multiple times. They had to create kind of separate folder in their project and call it. Should have a separate goal mode actually, then it'll be, use it as a separate package, and it'll solve the problem. I described it previously because if you look at your, go some, and this is go some from the Real project, for example, Google, Google. Has this amount of different, hidden or indirect dependencies. Why? Because, we have dependency, then we have another dependencies and we have another hidden dependency. And it could be different or like huge amount of layers or dependencies. And each of them may have their own go model these, specified versions. And this can end up, so in this case, guys was keeping, the contract the same, maybe adding new features, some, fixes, but the contract was the same and did, this didn't break anything. In our case, the break happened on the hand then yeah, we can proceed without code modifications, beef code, modifications, I would say. So unfortunately this is very common example and in my years in the company I have seen this multiple times so far for in different projects, and this happens over and over. If you go to the next example, I have these I called helper packages. This is also very common at some point, like when our project evolve, we, write and write in cotton and we find out we have some common parts we can reuse and move to the some, common library. First of all, inside the project for sure, and we create this helper packages or it, called, or whatever it may have different names, but idea the same. We applying the dry principle and we, remove duplications of the code. And in this example, I have package goer. And I have some helper. Helper has some random function, just for this, just example. and what we decided, okay, we are going to create another function inside our go, inside our helper package. And this, function is going to print some information about our goer. So it was going to, just. Print informatic the name with the age and position and then, yeah, we can call it in our main project. Oops. Ah, yeah. we can call it in our main project and we call it India. This is how it works. Unfortunately, place its slides in the wrong position. Yeah. What is the wrong, wrong in this example? So we have helpers and we have dependency to the goer object or another library in our project. And at some point it can lead to the situation when, we would need some helpers from the goer. Package and then what happens? We will create a link to their helper functions here and it'll create circle dependency infinity loop. I would say to avoid this, what we would need to create another helpers. Let's call it ER helpers, and this again, duplication of the code. So we are not solving situation, we are making it worse. Oh yeah, this is what I already mentioned. because when you create the helper, it's, and when you use a helper from one library, you, automatically adding this dependency to the goer package to all packages where you use it helper. And this creates some dependency tree, which at the end could lead you to the, Dependence, circle or dependency. Yeah. In my example, it's like very short pass to do this, but usually again, it takes a month, years until you at some point need touse something from the helper when you are inside the goer package and you can't because it doesn't work. So how to solve it, this is a easy example and I see such examples. Everywhere in the code. this info function should be part of our object. It should be met inside our object, so we can easily access information from the object. We even don't need to make this information public, so it could be these lowercase name or case age, and then we can print this information and this is how it used. This is better, I would say. You can argue with me, but this is better actually. and as I already mentioned, this is another problem I see in our code base a lot. So people are misconceptions, people not really thinking in advance and not applying common sense when they're moving some parts to the helpers and. Sometimes they don't see, this pattern, that these should be a method of the object. This shouldn't be separate helper function. There is no any reason to have it as a separate function, create all of these dependencies. So going back, I have, no, not going back, going user. I have another very common example related to the helpers because when you try to apply this. Right principle. You try to apply it almost everywhere. So you start to know, first of all, you try to remote applications inside your services. Then you see that another service, also has dependency, and you move in already some applications from different services. And so on and so forth. So this is example, we have a service, a and yeah, at some point we decided we need to create a helper package because we have some dependency, duplications in our code base. We created helpers. It work for us perfectly. but then, We have very similar service, b with very similar helpers, labor insight. And then we think, okay, there is no reason to do modifications in two places. Let's move out this, package to some, external one, and then they're going to reuse. But here again, we have the same problem with capacity. Maybe one team has enough time to work on the technical debris, removing these helpers from here or here. Maybe another team doesn't have the pen, capacity to do this. And we end up in situations like this one service using this helpers. Another service doesn't do this. Helpers. Or maybe even worse. And this I dislike more, that this service still has a library with helpers and some part of the code still use it, but also we already trying to know to migrate to the new solution and some of our, parts of our code using the new implementation from the external library. This is a huge problem and because implementations here may go into different directions, they may have similar name of the function with different implementations and different behavior, and also with different vulnerabilities Actually. But unfortunately this is not the worst case I have seen in Codebase. Actually, the worst one is this one. in this situation we have another service also implementing this library. And, for some reason, for example, we have another, library we are using from this service, or I don't know, some. Batch of services. And then it opens us the door to, use, helpers from this service as well. If we embedded it as a, not embedded, if we use it as a dependency in service a and then we have, three dependency with absolutely similar code. So our principle just fail it. Drastically failed. And I want to emphasize this is the real case. This is the real case I have seen on code base couple of times, which, made me very sad. Yeah. And yeah, I. About conclusions. your team acquires strong dependency management culture. Not only your team, it should be enforce it on the company level. It should be enforce it on the team level. You need to work with your product people to prioritize technical depth and remove, unnecessary dependencies. As you remember, I asked you at the beginning, can you name a couple of dependencies you have in your current project, and then can you name a couple of indirect dependencies you have in your project? Hope you can name some of them actually, but let's go. you need to commend dependency policies, so it should be written, it should be followed by the people. You should use this guidance to minimize. And the define what dependencies are, how they should be used. You should do regular dependency audit. you need to update versions. You need to detect and fix vulnerabilities. This is actually a super serious stuff because you may, use some library in your code base, which is, indirectly using another library, having, Vulnerability and you already expose it to the problem, because your application could be attacked and you even don't know why actually. So please update your versions, check your vulnerabilities. There are some tools to do this, so please follow the best practice and follow the common sense. don't skip imports. During PR review, this is a problem I, noticed multiple times with guys. so I mentioned this let go quickly back. Yeah, the, this slide, this happened because actually guys were ignoring these first sections of imports in the program. They was keeping on what could go wrong in the imports, but imports was saying, okay, the new library is going to be added to your project, and this library is very different from what you should use actually, if this won't be. it'll be solved way before when we face it, some circle dependencies in our code base or when, different parts of the code using look similar, implementation, but actually implementations are super different in them. And yeah, that's all from my site. I have a couple of slides, so yeah. we are hiring and if you want to help me to build, this good, dependency management culture in delivery Europe, please join us. We have couple of, positions in the company and yeah, if you have any questions or you want to discuss, this topic or any other topic, please don't hesitate to contact me, using any of this. Or websites and thank you guys. Have a nice day. Bye-bye.