Conf42 DevSecOps 2023 - Online

Ten Ways to Doom Your DevSecOps

Video size:

Abstract

Ten common organizational problems that virtually guarantee a less-than-successful DevSecOps adoption.

Summary

  • Almost every organization has managed to sabotage their own transformation or get in their own way. Here are ten policy that I run into time and time again that doom a DevOps effort. The next doom is using the cloud instead of leveraging the cloud.
  • Metrics should measure desired outcomes. Too many metrics in a complex system leads to management. Focus on one metric or a few metrics at a time. Don't be trying to measure many, many metrics at once. And the last doom is unrealistic expectations.
  • In summary, the ten ways to doom your DevOps slow onboarding using the cloud. Having change control board change advisory boards being afraid to fail. unchanging culture having unchanging policies. Setting unrealistic expectations. Let me know on the conference 42 discord if you've seen these or other common problems when you've been adopting DevOps.

Transcript

This transcript was autogenerated. To make changes, submit a PR.
You. Hello and welcome to ten ways to doom your DevOps. My name is Sheen Godimer. I'm a DevOps engineer with a government contractor called praeses, and I've been through a lot of DevOps transformations. Almost every organization has managed to sabotage their own transformation or get in their own way. So the transformation is virtually guaranteed to fail. Here are ten policy that I run into time and time again that doom a DevOps effort from the get go. Number one, slow onboarding. It takes days or weeks or months for a new team member to contribute meaningfully to the team. The problem with this is it makes them feel like they're a burden or it bores them, or it sets an expectation of pace that's really slow and they don't ever get really engaged and want to work faster and be truly engaged to contribute to the team. Most importantly, it shows them how important we consider them, which is to say, not at all, because we did nothing to prepare for them. Arriving on the team, have a laptop or a workstation ready, install the necessary tools, grant the necessary permissions, and then be prepared to pair or mob or ensemble code or work with them so that they really learn your process and your system and practice it ahead of time because they may not be used to it. Your team should have that as part of the culture so that they can onboard people quickly. The next doom is using the cloud. Not the regular cloud, but a pretend cloud. We run our data center in the cloud instead of leveraging the cloud. Well, that ignores the advantages of the cloud. So there are no advantages of the cloud. There are five essential characteristics of cloud computing as defined by NIST, but the one I see missing most often is self service. If you put your cloud resources behind a long request process or a complex ticketing and approval process, you aren't using the cloud. Then executives get frustrated because the cloud was supposed to make things faster and more flexible, but the organization is actively preventing that from happening by not actually using the cloud the way it was supposed to be. Similarly, doing agile instead of actually being agile, our organization has standardized the way teams will be agile, and there's no flexibility to change. Well, if you have a top down, mandated, unified process, you aren't doing agile. Agile is driven by self organizing teams. But you say we have a stand up for an hour every day. Exactly what we're talking about. Agile is supposed to encourage agility, right? And it's based on a couple of things, but one of the really strong pieces is remembering that it's individuals and interactions over processes and tools and responding to change over following a plan. And you can't have those if you're not a self organizing team. A great sign that you aren't agile is that you require status meetings and status reports and then status roll ups and then monthly summaries of all of those because there's too many status reports to really pay attention, to have your stakeholders join the standups, or at least the reviews, and that way they see what's actually going on and your team can actually get to agile. Doom number four, the change control board or change advisory board or review, whatever you call it. Every production change must be approved by a group that has no idea what we do. First off, you can't release multiple times a day if it needs approval from a group that meets multiple times a month. Right? Besides, what value are they adding in Devsecops? The process has reviews by the team and quality tools and quality gates all along the way so that our DevOps process itself manages risk, not some disinterested and uninformed group. The Dora report has shown us that ccbs increase risk, not reduce it. So show the rest of the organization that you have a process that you are disciplined in following, that you do pull request approvals, that you have quality gates, that you follow the principle of least privilege. And if there's a real question about what's being pushed out when, show them that you're using infrastructure as code and immutable infrastructure, so there's always really clear what's being pushed out to production, then you don't need the team or a board there to restrict what you're doing and how you're doing it. Next, being afraid to fail a failed experiment means we'll have a lot of explaining to do to management. Well, real Rogers has said good judgment comes from experience, and a lot of that comes from bad judgment. No failure means no learning. We're not going to get different results if we don't change what we're doing or how we're doing it. You need to work to the point where you're aggressively and ruthlessly protecting psychological safety and you're striving for a generative culture where learning, experimentation is just part of the way you work doesn't mean you just open up to the wild, wild west and anybody can do whatever they feel like. You can try to constrain the blast radius of any experiment you do. Make sure it only affects a small piece so that there's not much damage done. You don't lose a lot of time or a lot of goodwill. And be loud and public about learning from your experiments, both the good ones and the bad ones. Let other teams learn from your mistakes, and let other teams benefit from the things that you've shown that have worked. Plus, it lets the organization understand that you're getting to a different type of culture, a generative culture, where you protect psychological safety. Speaking of which, the unchanging culture will do anything it takes to get to DevOps. But we aren't allowed to change our culture. Well, bad news for you. DevsecOps is a culture, not a set of tools or practices. Right? And this has to be something that changes beyond just the team. It can't just be our team changes the culture, but the culture outside is terrible or restrictive or it doesn't like what we're doing because you're not going to get the benefits of DevOps if we restrict all the changes to only within the team. Now, you're not going to change the entire organization's culture at once, but maybe you can make some inroads and prepare the way and let them see that there are ways for the organizations to start changing because the DevOps team is showing them how it's working. If you don't manage your culture, your culture will manage you. Evangelize to other teams, make them want what our team has, show them how things are working, how things are going well, and maybe most importantly, find a champion, someone who's going to keep us from getting into trouble for doing things differently. They're going to help evangelize our culture. They're going to help show the rest of the organization how our culture is changing and how DevOps is helping similarly unchanging policies. We'll do anything it takes to get to DevOps, but we aren't allowed to change our policies. Well, existing policies are often designed around infrequent changes. Monthly, quarterly, annually, whatever it is. Right. And again, this needs to happen outside the team. We can't set policies just within the team and then have to deal with the rest of the organization's policies. When, as soon as we step out and have to deal with something that's not completely self contained, we're not going to get the benefits of DevOps. If we restrict change to only within the team, we're going to have to start making inroads to the rest of the organization. If there are policies that are standing in your way and you can't just ignore them, find out why that policy exists. Use automation in our process to generate evidence that we're going to meet the intent that we're going to constrain the risk. We're going to reduce the problems, right. And show how our process reduces risk via automation and via the process itself, rather than just somebody blindly following it for fear of getting caught, or worse, not following it and hiding the fact that they're not following it and just trying to get away with it. Too many decision makers. Every decision means meeting with multiple stakeholders to get consensus. If multiple people are responsible for a decision to be made, then no one's actually responsible. Remember the scene from office space? I have eight different bosses right now, so that means when I make a mistake, I have eight different people coming by to tell me about it. That's my only real motivation. It's not to be hassled. That and the fear of losing my job. But, you know, that will only make someone work just hard enough not to get fired, right? This type of too many decision makers, too many people having to have a say and approve things often goes back to being afraid to fail. Right? Again. Find a champion, highlight some of your early wins and make them want to take credit and responsibility. And that's going to lead you towards a generative culture where you're protecting psychological safety, where experimentation and learning is just part of the way you do business. Having too many metrics, we can't do that or it'll hurt our monthly or quarterly, our yearly numbers, right. Well, when a measure becomes a target, it ceases to be a good measure. Metrics should measure desired outcomes, right? We shouldn't just be blindly using any metric that we can find or what's available. And too many metrics in a complex system leads to management. Whack a mole. What you want to do is pick a target metric, improve that add process to protect it, to make sure it's going to stay where we got it to, and then pick the next metric. And slowly but surely focus on one metric or a few metrics at a time. Don't be trying to measure many, many metrics at once. It'll never work. And the last doom is unrealistic expectations. All of our problems will disappear as soon as we get to devsecops. Well, the actual case is DevOps won't make your problems vanish. In fact, it usually just shines the light on them and makes them more visible. The reality is we're going to need more discipline, not less. We're going to fail more often, not less. We're going to pick what we want to improve. We're going to work on it till that's better, and then we're going to pick the next thing. It's a long process. It doesn't magically wave a wand and fix everything. But if you set realistic expectations, people will see that you're constantly making progress towards improving. In summary, the ten ways to doom your DevOps slow onboarding using the cloud being agile having change control board change advisory boards being afraid to fail having an unchanging culture having unchanging policies having too many decision maker ers, watching too many metrics and setting unrealistic expectations. Please let me know on the conference 42 discord if you've seen these or other common problems when you've been adopting DevOps. Also, I'm other devsecops gene on X, Twitter, YouTube, GitHub, other social media. Reach out and let me know what you think. Thanks for coming.
...

Gene Gotimer

DevOps Engineer @ Praeses

Gene Gotimer's LinkedIn account Gene Gotimer's twitter account



Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways