Conf42 DevSecOps 2022 - Online

Building an engineering platform from the trenches

Video size:

Abstract

Engineering Platforms provide a great way to promote secure & compliant software delivery while reducing cognitive load for DevOps teams. Our talk addresses how to build a platform that entices and helps teams, rather than being forced upon, complete with experiences from the trenches

Summary

  • Hi, welcome to our platform engineering from the trenches talk. We'll take you through our experiences building platforms over the last few years. We have a story to share and some lessons learned so you don't fall into the same traps as we did.
  • In enterprise, organization means that you're always in a changing environment. This is where the platform dream and what the reality is. What you ideally want if you're building a platform is frictionless delivery of business value. To really come up with an integrated vision, you have to transform something that is transforming while the shop is open.
  • Most important thing for us is to be customer centric. Product thinking will help you design your platform in such a way that it focuses on two things. Always focus on capabilities rather than tools. Find your unique value proposition for your platform and that is unique to your context.
  • So of course, measuring your success is also very important. Start talking to people, for instance, to gather feedback. Measure things like customer engagement or employee satisfaction. Without these measures, you're not in the business of running a proper platform.
  • A unicorn topic is self service. These platforms where their value also is in how they can easily scale. How you can onboard more and more users without becoming a bottleneck yourself. Always be sure to remove waiting times if you go shopping in the supermarket.
  • How do you build trust and help people agree that we are going to build a platform? Focus on influencing the right people and the right teams so that you can get some traction. Be mindful of the pace that you are making change.
  • One of the things we really like to do is building a community around our platform. This is for knowledge sharing purposes. Also standardize on the way you communicate. We are really in the business of helping teams focus on core instead of context.
  • We seek two kinds of metrics. The metrics from the system and the metric from the people. We need to work more towards defining standards. The main goal that we still have is accelerated delivery of business value.
  • Keep improving the collaboration with your customers, helps them to make those technical decisions together. Of course, we really need to measure when we are successful. Goal is to accelerate the delivery of business value. Dora metrics are an excellent way to do this.
  • If you have any questions or want to discuss further, we're delighted to have the conversation with us with you. We'll be of available in the Discord channel of the conference and you can also always look us up on LinkedIn and just drop us a message and we'll discuss.

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Hi, welcome to our platform engineering from the trenches talk a deal of beautiful vistas and open sewers. My name is Gert. I work for Q Qxperts, which is part of CBI. I work as a product owner for continuous delivery platforms. My name is Hug Staplesva. I'm also a product owner for the CDAs platform and for pipeline enablement within national and Ayland. That's a big insurance company within the Netherlands. So in this talk, we'll take you through our experiences building platforms over the last few years. We've been doing that both in various roles, I guess engineering and, well, as the slide says, currently we're both working as product owner. So yeah, we have a story to share and some lessons learned so you don't fall into the same traps as we did. Yeah. Let's first take you through the agenda for this talk. We'll start off by setting the scene and then giving you an overview of our teams versus actual reality. We'll be talking rainbows and unicorns. Then the juicy part, tensions from the trenches. And then we'll spend a little time talking on where we see ourselves move. Yeah. Just to give you an insight what we're working on and which you perhaps can take as inspiration for yourself. So let's first set the scene. It's also important we both work in largely federated organizations, so we have lots of customers that are dealing with, that are internal customers and we are within a regulated environment as well. And that means that we're subject to rules coming from the financial sector, rules maybe around privacy. And most of the times we are in a central part of the organization where costs are actually very important so that you have to bring costs down and it is still considered a cost center and we are part of centralized it. So sometimes you get a bit of a feeling that you are from the ivory tower and you're telling everybody how you should do things. And finally, we are dealing with limited innovation or speed of innovation. This is where the platform dream and what the reality is. So we are dealing with both the vision there where we want to go and all the troubles that we have or the impediments that we have to face or that we have to resolve. Yeah. The interesting thing here is with these huge organizations is that especially these federated organizations, is that you have so many people there, each with their own ideas and their own agendas and their own goals, because in the end everybody is trying to achieve something. So everybody has his own goals and bringing that together. Yeah, that is quite complex, but also interesting. So if you look at it from a platform engineering perspective, what you ideally want if you're building a platform is frictionless delivery of business value. You see the picture with the person playing curling. That's the way I really see how platform engineering teams work, moving this broom so that the curling stone slides with almost no resistance all the way across the lane. So yeah, this perfect, seamless, frictionless way of delivering software, running software, whatever your platform is doing, that's the ideal. It should just be working. So what does it mean? It means also that the process that are baked in into the platforms, that they are actually aligned with reality. So it's not something theoretical from a slide, but people are actually behaving the way the process is designed, or the process is designed in such a way that it aligns with how people are behaving. Furthermore, ideally, obviously everybody has a clear and steady goal so that you know where people are moving and you also have a focus on business value. And there is always time, and this is a real engineering one, there's always enough time to build a proper solution that does things proper, not a hacky way, but in a perfect way. However, reality is, yeah, reality is sometimes something different. So first of all, priorities, what do you do first? How do you bring the most value to your customers who are actually your colleagues in such a way that you do the most important thing first? That also brings a lot of value to you as customers. Most of the times we already have a huge investment, for instance, in tools and the way they have set up. And to really come up with an integrated vision is also meaning that you have to transform something that is transforming while the shop is open. And of course, being in enterprise, organization means that you're always in a changing environment. Business department or business unit can change, stakeholders change means also priorities change. And somehow there is always the trap that you end up in the priority paradox. Yeah, because that's interesting because especially in these really large scale organizations, these federated organizations where I mentioned it earlier already as well, where people have their own goals, everything becomes a priority because everybody is trying to achieve his own goals. That's what matters at the end of the year, right? Yeah. So priority can be a challenging part. Customer relationships can be difficult as well, especially in a being organization. It can be very difficult to see who is your actual customer. You have people that are making noise, so you intend to move towards, you usually start moving towards these people. However, in reality, often there are a lot of other people using your platform as well, and then you have the people using your platform. So engineers. But then you also have usually internal security teams or compliance teams. Then you have architects who usually have an idea as well about what your landscape should look like. So identifying these customers is already hard enough. And even if you identify them, how do you communicate with them? If you're working in a small team, you can sit around the table and just talk to each other. But how do you do that in a large, decentralized organization? And then even if you've managed to fix that, how do you know that you have the right features for these teams? Again, if you're building something for someone who's sitting next to you, that's easy. But how do you do that when you're in this huge organization where people might be on the other side of the world? And then the last one that's also an interesting one is team specific. Own solutions. Usually when you come into an organization to bring a platform, there are some problems that have being existing for some time and teams, it is usually filled with smart people. So people try to build their own solutions. Yeah, they work around organizational problems. Exactly. And how do you have a good strategy to honor what they already have built, not breaking so much stuff and also letting them be heard as customers and let them feel, okay, this solution that I have created can be part of the larger solution that we roll out as a platform. Moving to our next topic, rainbows and unicorns. Obviously we have an idea of how this should be, what this should look like. So let's start there. First of all, the most important thing for us is to be customer centric. So we have to understand our customers. We have to understand how we collaborate with them. Of course we need to have a feedback loop on if they are using our product, how are they using our products? Are they happy with the product? And we want to do something that does not need force. So, meaning that we seduce our customers instead of having a force adoption. And of course we want to see our customers or our colleagues actually as partners within collaborating on creating this new platform. That's interesting because platforms are not a new concept, but this whole shift in how do we collaborate with our users, and especially the don't use force part, but treat your customer like a customer, make them happy. That is something relatively new but essential to get your platform successfully adapted. Yeah. And the second topic that ties into this is the product mindset. Product thinking will help you design your platform in such a way that it actually really focuses on two things at one hand, balance. So balance between the customer requests, which are usually opinionated on how something should be resolved. But yeah, also about how do you cope with the problems as a whole and also bringing it there in the dynamic of how do we solve this problem right now? Because usually a team comes towards you with an urgent problem that they want to have resolved right now, but how does that fit into your long term vision? And the second part is about real value. So it's really important to find your unique value proposition for your platform and that is unique to your context. Every organization is different, so that means that we're talking principles now, which we think are quite generic. However, how you apply them in your own organization can be completely different. So also coming back to that, if we look at the goals that we set for our platform, it's frictionless delivery of business value. But in a startup that can mean completely something different than in a heavily regulated environment. For startups, it could mean we all want to have speed where in regulated environments it could be security and compliant, where you offer lots of services that could be your unique value proposition for your product. Yeah, and as users on board, on your platform, your platform grows and this evolves also over time. So it's really important to every now and then, step back a little bit and think, hey, are we still on the right course? And the second point is, what's also really important is always focus on capabilities rather than tools. We all have our own personal preference regarding tooling, and we always think that the tool we use is the tool best, otherwise we wouldn't be using it. However, you need to find a balance there, or you need to find a balance and instead focus on what are we trying to achieve here, what are we trying to do? And since your platform is generic, how can we do it in such a way that might not fit with the ideal tool for a specific team, but rather empowers all teams using the platform to use this certain capability. So of course, measuring your success is also very important. And because this is the rainbows and unicorns part of our presentation, it's really interesting. So it's the obvious that you have to measure, but what do you measure and where do you start? So our opinion is here, let's start simple. So, measuring numbers of user, number of projects, but there are already a lot of measures out there that are really quite wonderful to use when it comes to delivering more technical platforms like the Doya metrics that are described in the book, accelerate or sdp metrics. And of course, it's also important to start with metrics for people. Start talking to people, for instance, to gather feedback, but also measure things like customer engagement or employee satisfaction to see whether people are really happy with the product that you are creating. Without these measures, you're not in the business, I think, of running a proper platform. I think your customers are your way to success because then you're bringing real value to them and you're also achieving your goal of delivering business value. Yeah, the interesting thing here is we coin these automated trend analysis terms, but this is not where you start. You can always start relatively easy, low level, just by starting to talk towards people and then only as your platforms grows in maturity, also grow in maturity of how you gather these measurements. Don't try to do it perfect at once. But be mindful though that you do want to be measuring. You do want to be measuring, so it should be integrated part of what you do. Okay then a unicorn topic is self service. These platforms where their value really also is in how they can easily scale. So how you can onboard more and more users without becoming a bottleneck yourself. So self service and enabling your teams to do stuff themselves, that's really important. So empower them and also make it easy to use your platform. We were talking about this product thinking earlier, people should not be annoyed to use your platform, but rather be happy because you're solving an issue for them, something less for them to worry about. Yeah, and last point, also, always be sure to remove waiting times if you go shopping in the supermarket. I don't know about you guys, but I already get annoyed when I need to wait for like a minute before I can do the checkout of my groceries, let alone if I need to wait four weeks before my new project is available for me on a certain platform. So remove waiting times. Yeah, I think that is one of the most important things that you can do to really improve the customer experience and also improve, for instance, employee satisfaction within your company because things are easy and teams can then focus on the things that on their own goals instead of dealing with the toil within the organization. Yeah, especially when working in agile environments, people, they are working with the small increment of work that they are working on. So if they need something and they get to you, they need something and otherwise they get stuck and you don't want to be yet another impediment. All right, so this is our magic rainbow unicorn world. Obviously reality is not that perfect all the time. Of course we're product owner, so we have to sell our platform. So we might say it is so. But I'm telling you right now, it's not always like that. What we will be sharing here is our insights of how did we start building platforms ourselves. First of all, we needed to build trust and help people agree that we are going to build a platform. And the interesting thing here is, well, at least my first pitfall was how do you build trust? I thought you do it by proving that you can do what you say. So it would be by building stuff, building new capabilities. However, reality shows that people, well, obviously they care about this new stuff as well, but it's way more important indeed that they trust you. So focus on the person. Yeah, indeed. So build up relationships with people. This is one important thing I learned as well. Build up this relationship. Tell the vision that you have on your platform, what your problems that you're going to solve. And also be aware that these things take a long time. You are building something foundational that is going to be used through all alti whole company. And that means that you have to talk to a lot of people to get your message across. And also what we notice is that you really have to focus on influencing the right people and the right teams so that you can get some traction. That is an important lesson that I have learned. Yeah. And the interesting thing is usually within your customer base you have a few people who are very vocal about any issues they run into, capabilities they want. However, be always sure to check that these people are not just some people who are trying to solve their own problems, but rather are the people that are really influencing what's going on in the organization. Because usually it turns out that there are some people, at least some people who are less outspoken, who are, because of the social role they have in the organization, are actually way more influencing your customers. So figure that out. Yeah. And one thing that I really wanted to briefly add as well is regarding the pacer steps. So we were talking already about this whole organization that is moving. But be mindful of the pace that you are making this change because depending on what your goal is, people need to be able to move with you. If you're being away a part of the organization and you're focusing on to a certain x percent of the organization that you want to move with you and you don't care about the rest, you can set a completely different pace than when you want to move your whole organization with you. Because then people indeed need to learn as well. Yeah. So a good example here is if you're, for instance, responsible for maybe some security and compliance checks, you can make it a direct stop of what a team is doing. But instead of educated teams, hey, this is what we're going to introduce. This is what you need to do. And maybe this is a channel where you can ask for help. And that is really how you pace yourself, but also move at the speed of the organization and also ensure that it's adopted by everybody. And otherwise you become this platform of frustration. And that is, I think, an important learning as well. Yeah. Know who your first followers are, who helps you create this movement. And the last one also to briefly stop by again about recognizing what's there. Be aware about. We mentioned it earlier already. Be aware of what is there already, and not because of the technology that it is, but also because it was created in a certain context. Some decisions were taken at some point in time. So it's always good to be mindful of this context as well. Even it might not be obvious from the solution itself because there are pitfalls that you can fall into again. So better prevent that when you then start delivering your platform, there are a few things that you need to think about. First of all, yes, we were talking about building this trust, but in the end, delivering something, how minor the increment is, is essential because it will allow you to start getting feedback on what you're building. So my personal take on this is being launching customers. So for every new capability that you're building, focus on find at least a single team in the organization that will be using this capability. By the way, good double checker. If you can't find a team that is willing to use your capability, you're probably building the wrong stuff. Yeah, so I completely agree with that. But often teams are already asking for capabilities and try to build it with them together. Offer them a channel to really communicate with you on every iteration that you deliver to your customer. Also have a discussion of there are some downsides for every feature that you deliver, make these decisions together because they have the advantage of being the first mover for that capability or that solution that you're offering. But you always have to keep every user of your platform in mind. In a sense, you become a bit of the devil's advocate of the organization when meeting the demand, maybe of one or two single teams. Yeah, what Hulk mentions here is really important. You're building this for the organization as a whole. So working together with this launching customer doesn't mean that you're going to do exactly as they say, per se, because you need to, also need to take into consideration this bigger vision and also the needs of the organization as a whole. So working towards an evolutionary model is always a practical approach there. Yeah. Two more things for this iterative platforms delivery. First of all, create an upward spiral. These processes, like Ilka already said, can be long, so it's good to get frequent positive reinforcement. You can do that by often releasing small new capabilities that make the lives of your users happy or better. And that's good for the spirit of your users, but it's also good for the spirit of your own team. About this team, get tough people in there. Well, the icon shows a rugby player. You can choose rugby players. It's not necessary per se, but the thing here is platforms are complex. You're building complex multilayered stuff. So you need a team that loves hard problems. Yeah. So indeed you need a team that really loves hard problems and likes to also solve those for everybody. So a platform is an excellent mechanism to scale out certain capabilities within your organization. But most of these problems are really hard problems that you have to solve. So you need people who can deal with it, or you need a team really that can deal with this context. So of course we always need some platform design principles. If you don't start out with principles, you will run out into problems with these principles. It's a bit of a guidance that you offer also the team that builds the solution, but it also offers a way to communicate to your customers a couple of principles that we saw that were essential for us. So first of all, is the authorization model. It is really the foundation that you have to build everything on. Without a good authorization model, especially in a regulated environment, you cannot build those integrated capabilities that you want to offer, but you also want to. One of the principles is reduce the number of touch points. The strategy that hitting and I both chose were a more reactive or event driven approach, so that people can use the tools as they were intended. But these tools, we do our magic on top of those tools or beside of those tools. So without the customer knowing that we do something to improve that capability that they're using. And of course we want to create working standards or define working standards. And there is always a bit of a trade off between simplicity. So how can we dumb down this particular process or this particular way of working versus is it really compatible with everything within the organization? And lastly, we don't want to break stuff or we don't want to fix things that work. There is always a bit of a history of using certain tools already in the organization and you're building a platform, maybe next or on top of those tools. So you don't want to break the experience or radically break the experience of the people who are using those tools. You have to have a really soft touch so that people keep enthusiastic about your platform and that you're really offering something more than the tool itself. It's really about the integrated solution that you're building and solving those complex problems for those teams. Yeah, and this is also again, where this frequent feature or capability delivery comes in as well. If you take small steps and take them often, then this is a journey that you can make together with your users, rather than throwing helps and helps of unwanted change upon them. Also, one of the things we really like to do is building a community around our platform. This is actually for multiple reasons. First of all, for knowledge sharing purposes. What we figured out is, especially if you work in large organizations, often multiple people are looking for the same answers. So if you have a centralized community, it becomes easier for people to find their answers. And also what can happen is that actually users start help each other out. As one of these communities grow and people become more involved in it, they are also more willing and almost automatically starting to share their own knowledge and also their own experience. And if they don't know something, they will start asking questions. And these questions are interesting for you as a platform development team because it will give you insight in what's going on with your user. So if people ask the same question over and over again, don't be annoyed with it, but learn from it. Hey, why are people asking this over and over again? Is our documentation not clear enough or is there a hiccup in the system? Or is the experience of our platform as a whole lacking? Yeah, use it as a moment to learn. Yeah. So very good way to receive feedback on what you're building and what you're delivering to your colleagues. Yeah. And then last one, also standardize on the way you communicate. I personally have a very strong preference of single chat channel. Either be in teams or slack or whatever tool you're being, moving all the communication there, because it will allow teams to understand where they go with their questions and go to meet the community. And also always be mindful that you're open communicating here. So for example, one of the patterns that I really focus on is if the question pops up, answer it in this channel. Yeah, we sometimes even go as far. If you get a direct question from one of your colleagues who is using your platform, ask them to repost the question in decentralized channel, because not only you are going to learn from it, you can see whether others are also experiencing this problem, but it's also a great opportunity to help inform others how you can solve a certain problem within the platform. And lastly, this will also help again build trust in your platform because it shows to your users that you care about their questions and care about their opinions and see that you're engaged and that you're actually intending to work with them to build the best experience possible. Yeah, I think it's a good lesson to learn as well to really think about community here. So we, as platform team or platform team owners, we are really on a healthy diet of complexity. And when you go back to one of our first slides, we said we want to have friction delivery of business value. I think when we're eating complexity, and especially within the enterprise, we can eat a lot of complexity and make sure that every feature team, for instance, is able to deliver as fast as possible. And yeah, there is something called core versus context. We are really in the business of helping teams focus on core instead of context. And again, listening to our customers, receiving feedback are essential here and also delivering a solution that really helps to fulfill those goal. Yeah. And in addition, also be aware that if you're starting to reduce context, so if you are starting to abstract away complexity for teams in the organization, be aware that these teams are no longer aware of this complexity that there is. So that means that always think about, hey, is this something that we are abstracting away for them, but do we still want them to be aware of? In which case you also need to make sure that you explain what you're doing or is it safe for the teams to, well, just rely on it? Is it just like electricity coming from an outlet? Yeah, exactly. It's there. You don't know how it's generated, where it comes from. It's just there. And that is, I think, the ultimate goal. I think also for platform teams to just be there and help teams deliver business value. Yeah. And then that also means that sometimes the biggest compliant you can get is zero compliant. If you don't hear anybody. Well, either they're not using your platform at all, but most probably, at least from your other metrics that they are using it. And if there's no complaint, actually it could very well mean that you're doing quite a good job. All right, so this is our experience so far. Let's have a brief look at where we are moving towards with our own platforms. First of all, it's improving metrics. We actually seek two kinds of metrics. The metrics from the system. So this are usually technical metrics, and then you have the metric from the people, which is the complaining we're talking about, or, well, other kinds of opinions. What we're working on is shifting there from the relatively easy to gather flat metrics like number of projects or number of pipelines run something like that towards metrics that are more interesting from a business success perspective. So again, here you can think about, for example, Dora metrics or software delivery performance metrics, but also, for example, about instrumentation of capabilities. So how are people actually using this platform? And then on the people side. Yes, do surveys. Yes, ask this feedback in your community, but also think about stuff like customer engagement or net promoter scores. Yeah. So especially the net promoter scores are quite an interesting thing because it's standardization, how you measure, how building people are to promote the product that you deliver. And especially when your platform is growing or you want to grow it within the organization, or you want to have it adopted by the organization, you need to have these scores correctly in order so that people are really sharing the message that you have within the organization. And what we mentioned before is we are both within a federated organization. It's very large. So how do you reach everybody, especially when others are going to promote your platforms? That would be great. So also we need to work more towards defining standards. And we are aware that we are on a journey. And the environment changes, team changes, the organization changes, and we are set on a journey, and it's not one time actions that will follow. So we are adopting a more evolutionary model. And it's important to keep your principles in mind. The principles are the basis for your guidance. And the main goal that we still have, if you look one year back today or five years in the future, is accelerated delivery of business value. And this is where we bring value to our organizations. Of course, there is always the balance between, okay, this is what we envision, this is the direction where we want to go to. But there is also something that is the daily reality. Things are not perfect, and this means that you have to deal with impediments, your dependencies, but it's also to keep on a focus on your ability to adopt and what your customers need. And there, it's also what we previous touched upon this journey part. It's also part of this journey. Yeah. And here again, the topic of evolution comes in. So if you are defining a standard, make sure that you not only think about right now, but also how is it going to work in future, for example, with the authorization model. Yeah. One thing you know for sure is that there will be tools in your platform phased out and there will be different tools and they will have their own access model, for example. So be aware that you design this in such a way that you can evolve with it and can make your platform involve. Yeah. And also make it as easy for your customers as possible. And that is, I think the next part that we have on our slide is really about keep improving the collaboration with your customers, helps them to make those technical decisions together. Often our customers are quite technical and make them part of the conversation that you're having and continuously validate whether the technical direction that you are going into also meets what your customers can deal with. But also, it's also good feedback. Maybe I need to simplify something. Of course, we mentioned already a couple of times before, we really need to measure when we are successful. So the goal is to accelerate the delivery of business value. I think Dora metrics are an excellent way to do this. But also think about, for instance, employee engagement or net promoter score to see if you can have the organ or see whether the organization can help you to scale your platform. Yeah, so that's it. Thanks for watching. If you have any questions or want to discuss further, we're delighted to have the conversation with us with you. So we'll be of available in the Discord channel of the conference and you can also always look us up on LinkedIn and just drop us a message and we'll discuss. All right, thanks. Thank you.
...

Geert-Hinke Addink

Product Owner @ Qxperts

Geert-Hinke Addink's LinkedIn account

Hylke Stapersma

Product owner CDaaS @ Nationale-Nederlanden

Hylke Stapersma's LinkedIn account



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways