Conf42 Chaos Engineering 2021 - Online

Shift up: Continous Security and feedback loop in production

Video size:


DevOps engineering culture demands deploying code at lightning speeds. And speed equals carelessness. And carelessness may lead to breach.

This talk is an introduction to shift up paradigm, think of it as an extension of shift left but a culture that only strives in production. Shift up enables an organization identify, and remediate insecure code and address any security gaps within infrastructural stack in seal-healing and iterative manner. To achieve this end state an organization needs to perform defensive dynamic security testing and test configuration, and system failures against A/B units. These exercises helps validate effectiveness of production’s layered protection, which is responsible to protect application code and most importantly customer’s data. And last but not the least, building capabilities to identify external-facing assets in continuous manner and monitor it through out its existence. Enabling an organization with a feedback loop between *AST tools (SAST, DAST, IAST, MAST) and layered defenses in production. Further arming them with a protective shield against ever-evolving attacks and ultimately gaining IT utopia!


  • Certus Cybersecurity provides security services to various Fortune 500 companies. CTO Swapnil Deshmukh says there is a gap when it comes to communicating between non production and production networks. He suggests a continuous feedback loop between the two.


This transcript was autogenerated. To make changes, submit a PR.
You. Hi everyone, my name is Swapnil Deshmukh and I'm the CTO and co founder of Certus Cybersecurity. Certus Cybersecurity is a security consulting company and we provide security services to various Fortune 500 companies. And a common theme that we have seen across the board is that we have new terminal logies from a non production network standpoint itself, such as shift left, where all the security continous are being passed over to either developers or are being tool based. And they have a certain set of policies that needs to be adhered to during the entire code of software defined lifecycle. And on the other hand, we have traditional technologies like red teaming exercises that happens in addition to that. Now there is chaos engineering and there is mitre attacks. There are apt relevant exercise as well. So something very narrowly focused than what Mitre would. And the reason for that exercise itself is to not be very intrusive from the testing protective itself, but just to do a pressure test to identify how vulnerable an infrastructure would be. And both technologies in itself, in its own right, are something that we certainly need. But what we see is that there is a gap when it comes to communicating from what learnings we have from shift left perspective, and also the learnings that we have from our red teaming, or from mitral attacks, or from chaos engineering itself. And since there is a gap, the non production tends to have similar set of issues that gets trickled into production itself. And all of these attacks are mostly point in time. And what that means is that if I do my testing today, but if you push the code tomorrow itself, I would not be able to test it because the time has passed from a testing perspective itself. So all of these engagements are not continuous testings per se, they are point in time testing. As a result of that, if after the test has been done, someone pushes a new code to the production for the things that were already validated in non production itself, it tends to have these insecurities again passed over to the production network. As a result of that. We strongly recommend that people must be thinking, should be thinking about how can they integrate continuous feedback between these two silos that we have non production network and production network, and the testing that happens between the two from a security standpoint. And that's what we are proposing from a shift up perspective. So shift up perspective, we are talking about continuous feedback loop. And that feedback loop essentially means that the production network, whenever it's discovering new things from a policy standpoint, that they are updating within their ecosystem itself should automatically get trickled down into non production as the policies that they need to enforce as well. And it could happen in two ways. Either it can have similar infrastructural and the infrastructure talks to each other from an API perspective itself, where these policies get pushed and then these get validated during the non production testing that is performed through pen testing for example, or could be dynamic testing like dast for example. So it gets validated there. If it's application level testing itself, then the non production learnings that they have would get trickled down as a policies into the production. So this talking back and forth itself is what we are proposing. And we have seen that that has been very, very effective for a few of the companies that we have worked with in the past. However, there is a course, there is a journey that company needs to embark whenever they are thinking about integrating DevOps or devsecops, or integrating security and compliance within DevOps pipeline itself. And the journey needs to be first it needs to be done manually so that you can have a lot of historical data about all of these policies, all of the different security controls that one should be considering, and then looking at those policies and translating that into either of the networks to give you an anecdote around it. We had a WAP solution that these company was using and the VAP was obviously automatically updated thing the attack trees. So whenever it will identify that there is a certain set of attacks that is happening, it will look at what are the new regexes, what are the new attack patterns that an adversary is trying to attack them with. And it will, behind the scenes start updating those regex in production and in the non production itself. They did not have the WAF at any given point in time. As a result of that, those policies were never getting updated. However, the application was very resistant against SQLI injections itself or SQL injections because it had its own set of rule sets. One fine day they discovered that Wap was completely misconfigured from a SQL injection standpoint. As a result of that, an adversary had passed through into the application layer directly. Without these, new WAF can directly talk to the application itself and could have tried to perform SQL. Now, fortunately, the company had already thought about it and started integrating that into the application because non production never had the WAF or the luxury of WAF and testing always would have found it. But if you have to set up the shift up properly, then you should mimic the production network as it's so the WAF should have been there, then you can perform two set of testings where you have a test with the WAF and one test without the WAP. And you can see how resilient your application is against SQLI by doing that exercise itself. But then by having both talking same terminology from layer defenses perspective itself, it becomes very easy for us to perform more set of testings when it comes to penetration testing itself and non production. Because even if we take down the run production because these sqlites up, it doesn't have any impact on production. As a result of that, all the learnings that we have during that exercise itself and all the new regexes that it has created, they both should be syncing together on those regexes. As a result of that, the policies would be updated on both ends are supposed to be updated in one there was a company that wasn't doing SQLite properly, very very big name in telco, and in this particular company itself they fortunately had a bug bounty program where we were able to identify a lot of SQLis by just using this as an attack pattern itself. So we were able to penetrate into their WAP regex and directly communicate with it because they never had either shift up or the continuous feedback from their development team itself back to these production. So I hope this concept of integrating chaos engineering and integrating with the learnings that we have from the production itself into non production and vice versa in a continuous fashion would help create a very resilient security system for your company. Me. And if you have any questions or if you want to reach out to us, please do not hesitate in reaching out to our either company which is Certus or our email address is info

Swapnil Deshmukh

CTO @ Certus CyberSecurity

Swapnil Deshmukh's LinkedIn account Swapnil Deshmukh's twitter account

Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways