Hello. Welcome to our session the power of AWS tax, in which we're going to tell you how we managed to implement the twoperson approval on top of AWS IAM and implemented a scalable ABAC solution around that. My name is Yoav Yanilov. I'm the solutions architect for Axiom Security, and my co host Itamar Bareket, CTO of Mobimatter, is not here today because he just moved with his wife to Dubai. The agenda for today is to first realize that ABAC is broken, fixing it, and then profit. But the real agenda is asking ourselves, can someone delete an obvious cluster in my organization? And if so, who? The answer we all like to give is no one. But come to think of it, there is some sort of an admin user. Maybe it's behind some breaklast protocol, but are we really sure that we are defending this user the way it is supposed to be defended? This is going to be an advanced IAM session, so we are assuming that you are familiar with AWS IAM. And in case you're not, here's a very quick refresher. Tags are essential to ABAC in AWS, and each in flight request in IAM carries over metadata about the tags involving the principle, which is the identity making the request. The request itself can have tags also, and the resource that the request addresses can have tags as well. And in addition to the three of those, there's also metadata that concept the keys for the tags that are being passed in the inflight request. This is very important and we'll see why in a bit. But first things first. How do you enforce permissions on can account level? This is something that AWS has for a while. It's called an SCP and just an IAM policy. Only you apply it on the organization's level, meaning you apply it to an organizational unit or several accounts, and it applies to each and every role and user and IAM principle in that account. Let's talk a back assuming I want to make sure that no one can delete an RDS cluster in my organization, I can have some sort of an SCP that looks like this. It says you must be tagged with an isadmin. True, the principle must be tagged in order to run any delete operations on the RDS service. And it seems fine enough, but who'll guard the guards here, right? We have to implement sort of logic that says you must be in the club in order to get other people in the club, meaning you must be an admin in order to be able to tag someone is admin equals true. Without that logic, some sort of an iM user or privileged role may have a path that might not be direct, might go around like passing some sort of a role to a lambda or something, but it will eventually be able to tag some other role or user with admin equals true and bypass the original constraint that the SAP has. So this is why ABAC is sort of a mess in AWS. And to add more fuel to that fire, we have CTO scale that entire logic, meaning we're not going CTO have just one SAP in organization. We're going to have a couple, or maybe tens, hundreds. And writing that same logic over and over again is error prone and tedious. What we have here is a classic scenario of data plane, which is essentially who can do what depending on tags, and the control plane, which is sort of like the integrity logic. And this is how we perceive these two things. There's the access control logic and the tagging integrity logic. The first one is plain simple and the other one is kind of tricky to implement, but we want to be able to scale the tagging integrity for our scps in organizations. So let's take a path based approach here. Let's say that Alice is granted write access to Alice. She can then access everything within that path, that grant area. So Gitmes. But she does not have access to Bob's Humfolder and system. And if you translate this into IAM, if we're saying that Alice's grant area is home Alice, then she can add or remove tag keys, right? Meaning tag or untagged, any tag that matches the home Alice prefix. And then we have the chicken and an egg problem all over again. So if it was Unix tagbased system, you can imagine that there's a file called Grant path under mega, and no one can write to that file. And that file contains the prefixes, the grant areas for each and every one of the users in the system. And when we come to translate this into IAM, we realize that there is no centralized grants file. And that's why we have to make a unique special tag key that's called path, which denotes the grant path for each and every one of the principals that are playing this game of control tags. And of course no one has any permission to add or remove tags undermeda. So how does one go about doing something like that? We'll say that in a bit. Introducing control tags. Control tags is a scalable tagbased control plane for tagging operations on top of AWS IAM. And we start off with saying something. You know what control tags are? Tags that start with SWCDL. SW stands for similar web. This is where me and IAM worked when we developed this methodology. So SWCDL denotes control tags, and other tags are not without prefix. If we had to pick the most important slide in this presentation, it's this one. This is the entirety, more or less, of the control tags SCP policy. So what are we looking at here? The first palette is the sort of a club thing. You have to be in the club in order to invite other people to the club. Meaning that if you're trying to tag something with SWCTL and you don't have made a grant path, you are denied. Right? And the second part? The second statement says that if you are tagging or untagging a control tag, that control tag must be an extension of the grant path, which you already own, or one of our organization's legacy tags, which is kind of critical. If you want to use infrastructure AWS co systems like terraform, and have that play nicely with this mechanism, then we get what seems to be a very simple recipe to ABAC at scale. You have the control tags policy, which you then associate or connect with an AWS account or organization unit. And then you write your own featured SCP in a different policy, and you attach both of them to the target account or ou. Let's revise the bad example and see how much simpler it is. Look at it. There's only addressing of the tagging access control logic. Nothing about the integrity here, because the integrity is implicit. It is derived from the guarantees of the control tags policy. By specifying the admin RDS, we realize that you must have a grant of either the first one, v one admin RdS, deleter, v one admin, v one, or the root in order to be able CTO beat RDS resources. But can we make it even more secure? What if we must have some other admins approval if we want CTO delete an RDS cluster? Right? Seems much more secure. And now we arrive to the concept of approvals in the cloud. In other systems like GitHub, it's very popular to have an approval button, and if someone else did not approve your tool request, you're unable to merge it. But in cloud environments, it's kind of hard CTO think about it, because destroying an RDS cluster is one single API request, so it either succeeds or fails. There's no multi step process here. Our first use case is very simple, very naive. We don't want anyone to be able to delete RDS resources or s three buckets or even vpcs without an admin's approval. A second admin's approval. And first and foremost, we want to convince you that it works. So let's watch a demo. What we have here is two terminal sessions. The upper one is mine and the lower one is ecomas. And now I attempt to look for s three bucket with my surname. There it is. CTO. Meet me if you can. So I'll try to delete that bucket, see what happens. It fails, right? It fails because I do not have an approval from IAM. Now Itama runs a script which approves my identity. And when I attempt to delete a bucket again, let's see if it works. It does. No error. Let's search for that bucket and make sure that's not here. It is not there. So what just happened here? In an approval pattern based on AWS Im, Alice approves Bob's role by tagging his role with a two PA ticket tag. So it's just an IM tags that's used to carry over enough metadata in order to convince AWS iam that Bob acts with Alice's approval. Let's see the anatomy of a two p ticket. So the part of the key is, of course, sits under the admin, meaning it's an admin only feature. Admin, two PA ticket. And the value for that tag consists of three areas. The first one denotes Alice's identity, the third one, Bob's, and the middle can have some sort of a payload which is kind of useful. We'll see later on. Why. Now we're going to deep dive into the control plane of the two PA SCP itself. Hold your breath. The first thing first we have to have some sort of an organizational chokehold around who can even assign source identities. Source identities in Aws are sticky meanings. Once they're set on a role session, they cannot be unset or even removed. And here we request some sort of control tag under meta, again, so not accessible to admins, which denotes identity broker equals true. And we're going to tag each and every one of the relevant OIDC or Sam'l providers or even our homegrown identity brokers with that role, which is going to guarantee that if you attempt to set a source identity without that tag, you are denied. And also, we have a second statement here saying that you cannot do any tagging operations on IAM if you don't have a source identity attached, meaning you have to have a source identity which has been dispensed to you by an allow listed identity broker. The second part deals with antireflexive integrity. What does that mean? It means that the receiver section, the last one, meaning that it can all be set to the identity of the principal making the request. Right? Meaning I cannot give an approval ticket for myself. I can give an approval ticket for any other source identity in the system, whether it exists or not, but never to my own. The third part deals with forgery, meaning I cannot go about and forging a ticket, say, to Bob, and telling it's from Eve and not myself. This is crucial because those typing operations are audited in Cloudtrade, so we want to always have visibility into who gave the ticket to whom. In summary, it's an admin only feature. It provides us with identity integrity, anti reflexive integrity, anti forgery integrity, and it is payload bearing, meaning we can specify a CTO, a Unix timestamp if you may, for that ticket, and have an external retention process remove the tickets when the time is due. And turns out it can't all be nice and simple. IaM, we need to write some code for that as well. So what we have here is the architecture for that. It looks kind of complex, but it's fairly simple. We have a lambda function that wakes up every 30 minutes, scans the entire organization, and just removes two PI tickets from each and every role or user where they are applied to, if their TTL has elapsed, or if their TTL expires way into the future, more than 4 hours into the future. Right. Now that we got the control plane out of the way, let's look at the data plane, and it's surprisingly short. This is the entire data plane for that SCP. Let's read it. It says you deny the actions of deleting an s three bucket, an RDS instance or cluster, unless you have an approval ticket which is assigned for your own source identity. And that's it. That's the solution here. But then we looked at this condition and we said, hey, this looks very relevant. Let's copy and paste that back into the control tags SAP itself. So what does it do in that context? In that context, what it means is that you can escape the guardrails of the control tags themselves by having another admin's approval. Meaning that if two admins collaborate, they can tags or untag things outside the grant path. So I'm not just bound to v one admin anymore, I can tag v one, meta or v two hello world. Whatever I like a second use case, has more of a granular nature to it. We said, great, we can restrict sensitive actions, but now we want to restrict sensitive actions on some resources. And we refer to this features as two pa seals, which we also like to call two PA seals. And the way it works is, let's say I have an AWS secrets manager secret and I want to tag it with seal equals total. And once I do that, I want an admin room to be able to describe that secret, but not to reveal it, and never to remove the seal. Whereas the same admin role with a two PA ticket can describe the secret, reveal it, and even remove the seal. So let's see how it's done. But first, let's reflect about seals for a bit. We're saying that sealing something totally, it's useful for manually covered keys. That's how we store vault keys in our organization. And sealing trust relationship is kind of useful against privilege escalations. And we'll see why in a bit. Let's explore trust relationship seals. Right? So the first part says that you cannot bypass the seal without a two p a ticket. That shouldn't be surprising, right? If the resource is sealed with a deny trust update and you don't have a two PA ticket, you are unable to update that roles. Assume a policy. And the second part here is the seal lifecycle. Right. It means that you cannot change the seal or delete it without a two PA ticket. But press not, the end is near. Let's see just how useful those trust relationship seals are. This is what we call the two PA trust relief pattern, and it gives us two guarantees. The first one is you cannot assume that role without an approval, without a two PA ticket. And the second guarantee is that you cannot make any changes to the trust policy without an approval. That's a guarantee of the deny trust update two pa seal. So here we have that trust policy that allows the admin role to assume it, but denies any kind of action if there is no two PA ticket attached. And let's call that role super admin. And at the bottom of the slide, you see that there's also a tag on it, a two PA seal with a value of deny trust update. Using that trust relay pattern we just saw, we can extend the process of approval to other systems like AWS eks. The first thing we're going to do is make sure that the super admin role is assigned to systems masters on our EKs cluster, and that the regular admin role is assigned to some sort of auto group. And it's important to denote here that this group does not have the equivalent of IAM permissions within bks, right? So regular admin cannot make new Kubernetes roles and Kubernetes role bindings. The second part would be to provision said super admin user with the trust policy and the tags, the seal that we just saw and that's it. That's how you relay the trust to an audience systems, the same way that it has been done for eks can be done for a hashicop vault. Let's play a game of cat and mouse for a second here. Let's suppose we have packed admin in the system and that packed is eyeing data within a sensitive bucket, the system is protected. The bucket has a two PA ticket God around it, meaning the bucket policy cannot be changed without a ticket. It also specifies that only the sensitive role may access it. The sensitive role has a two PA seal around it, and the super admin of the system has a two PA seal and requires a two PA ticket to assume. That's a guarantee here. And the first thing the hacked admin is going to attempt to do is get object or put bucket policy, but both of those are denied by the bucket policy, which kind of hardens the bucket. Then the attacker may attempt CTO, assume the sensitive role, or pass it on to Lambda or EC two machine where it can privilege those permissions and gain access to the bucket. But it is going to fail because it is denied by the trust policy which only allows access from the eks path. Then the actor will attempt to change that trust policy and it is denied by the two Paco because you must have an approval ticket if you want to change the bucket policy. If you want to change the sensitive roles policy, and then the attacker might attempt to exact into the EKs cluster he's going to try to edit the AWS off config map which is in the Kube system namespace, or even create role bindings or cluster role bindings in order to elevate its own permissions. But if we made sure that the regular admin role does not inside the Kubernetes cluster does not have access to those APIs, those will fail as well. Then the hacker might attempt to assume the super admin role and continue from there to the eks cluster, but it's unauthorized again because the admin role requires a two k ticket in order to assume it. The last thing an Acura might attempt to do is create his own Sam'l provider or IDC provider and then create two roles and have one role attempt to sign an approval for the second role. But that will fail as well because the sample provider or IDC provider that the hackers just created does not have the identity integrity clause assigned to it. That identity broker equals true tag. So the two PA SCP is going to deny that? Yeah, so it's all control tags, always has been. We're using it in production for more than two years now. It's used by a team of more than ten admins, and we're using it for basically anything. It made the process of creating new security features on organization very agile and seamless because we never have to stop and think about tagging integrity. But thank you very much.