Transcript
This transcript was autogenerated. To make changes, submit a PR.
Hello. Welcome to our session the power of AWS tax,
in which we're going to tell you how we managed to implement the twoperson approval
on top of AWS IAM and implemented a scalable ABAC solution
around that. My name is Yoav Yanilov. I'm the solutions architect
for Axiom Security, and my co host Itamar Bareket, CTO of Mobimatter,
is not here today because he just moved with his wife to Dubai.
The agenda for today is to first realize that ABAC is broken,
fixing it, and then profit. But the real agenda is asking ourselves,
can someone delete an obvious cluster in my organization?
And if so, who? The answer
we all like to give is no one.
But come to think of it, there is some sort of an admin user.
Maybe it's behind some breaklast protocol,
but are we really sure that we are defending this user the
way it is supposed to be defended? This is going to be an
advanced IAM session, so we are assuming that you are familiar with AWS
IAM. And in case you're not, here's a
very quick refresher. Tags are
essential to ABAC in AWS,
and each in flight request in IAM carries
over metadata about the tags involving the principle,
which is the identity making the request. The request itself can have
tags also, and the resource that
the request addresses can have tags as well.
And in addition to the three of those, there's also
metadata that concept the keys for the tags
that are being passed in the inflight request. This is very important and
we'll see why in a bit. But first things first.
How do you enforce permissions on can account level? This is something
that AWS has for a while. It's called an SCP and just
an IAM policy. Only you apply it on the organization's level,
meaning you apply it to an organizational unit or several accounts,
and it applies to each and every role and user and IAM
principle in that account. Let's talk
a back assuming I want to make sure
that no one can delete an RDS cluster in my organization,
I can have some sort of an SCP that looks like this.
It says you must be tagged with an isadmin. True,
the principle must be tagged in order to run any delete operations
on the RDS service. And it seems fine enough,
but who'll guard the guards here, right? We have to
implement sort of logic that says you
must be in the club in order to get other people in the club,
meaning you must be an admin in order to be able
to tag someone is admin equals true.
Without that logic, some sort of an
iM user or privileged role may have a path
that might not be direct, might go around like passing some
sort of a role to a lambda or something, but it will eventually be able
to tag some other role or user with admin equals
true and bypass the original constraint that the
SAP has. So this is why ABAC is sort of a mess
in AWS. And to
add more fuel to that fire, we have
CTO scale that entire logic,
meaning we're not going CTO have just one SAP in organization.
We're going to have a couple, or maybe tens,
hundreds. And writing that same logic
over and over again is error prone and tedious. What we have
here is a classic scenario of data
plane, which is essentially who can do what depending on tags,
and the control plane, which is sort of like the integrity logic.
And this is how we perceive these two things. There's the access control logic
and the tagging integrity logic. The first one is plain
simple and the other one is kind of tricky to implement,
but we want to be able to scale the tagging
integrity for our scps in organizations.
So let's take a path based approach here. Let's say that
Alice is granted write access to Alice.
She can then access everything within that
path, that grant area. So Gitmes.
But she does not have access to Bob's Humfolder and
system. And if you translate this into IAM, if we're
saying that Alice's grant area is
home Alice, then she can add or remove tag
keys, right? Meaning tag or untagged, any tag that matches
the home Alice prefix.
And then we have the chicken and an egg problem
all over again. So if it was Unix
tagbased system, you can imagine that there's a file
called Grant path under mega, and no one can
write to that file. And that file contains the prefixes,
the grant areas for each and every one of the users in the system.
And when we come to translate this into IAM,
we realize that there is no centralized grants file. And that's why
we have to make a unique special tag
key that's called path, which denotes
the grant path for each and every one of the principals that are playing
this game of control tags.
And of course no one has any
permission to add
or remove tags undermeda. So how does
one go about doing something like that? We'll say that in a bit.
Introducing control tags. Control tags is a scalable
tagbased control plane for tagging operations on top of AWS IAM.
And we start off with
saying something. You know what control
tags are? Tags that start with SWCDL.
SW stands for similar web. This is where me and IAM worked when we developed
this methodology. So SWCDL
denotes control tags, and other tags
are not without prefix.
If we had to pick the most important slide in
this presentation, it's this one. This is the entirety,
more or less, of the control tags SCP
policy. So what are we looking at here?
The first palette is the sort
of a club thing. You have to be in the club in
order to invite other people to the club. Meaning that
if you're trying to tag something with SWCTL and
you don't have made a grant path, you are denied. Right?
And the second part? The second statement says
that if you are tagging or untagging
a control tag, that control tag
must be an extension of the grant path,
which you already own, or one of our organization's
legacy tags, which is kind of critical. If you want to use infrastructure
AWS co systems like terraform, and have that play
nicely with this mechanism,
then we get what seems to be a very
simple recipe to ABAC at scale. You have
the control tags policy, which you then associate
or connect with an AWS account or organization unit.
And then you write your own featured SCP in a
different policy, and you attach both of them to
the target account or ou. Let's revise the
bad example and see how much simpler it is.
Look at it. There's only addressing
of the tagging access control logic. Nothing about the
integrity here, because the integrity is implicit. It is derived
from the guarantees of the control tags policy.
By specifying the admin
RDS, we realize that you must have a
grant of either the first one,
v one admin RdS, deleter, v one admin, v one,
or the root in order to be able CTO beat RDS resources.
But can we make it even more secure?
What if we must have
some other admins approval if we want CTO delete an
RDS cluster? Right? Seems much more secure. And now
we arrive to the concept of approvals in the cloud.
In other systems like GitHub, it's very popular to
have an approval button, and if someone else did not approve
your tool request, you're unable to merge it.
But in cloud environments, it's kind of hard
CTO think about it, because destroying an RDS cluster
is one single API request,
so it either succeeds or fails.
There's no multi step process here.
Our first use case is very
simple, very naive. We don't want anyone to
be able to delete RDS resources or s three buckets
or even vpcs without an admin's
approval. A second admin's approval.
And first and foremost,
we want to convince you that it works. So let's watch
a demo. What we have here is two terminal
sessions. The upper one is mine and
the lower one is ecomas.
And now I attempt to look for s
three bucket with my surname. There it is. CTO. Meet me if
you can. So I'll try to delete that bucket, see what happens.
It fails, right? It fails because I do not have an approval
from IAM. Now Itama runs a script which approves
my identity. And when I attempt to delete
a bucket again, let's see if it works.
It does. No error. Let's search for that bucket and make sure that's
not here. It is not there.
So what just happened here?
In an approval pattern based on AWS Im,
Alice approves Bob's role by tagging
his role with a two PA ticket tag.
So it's just an IM tags that's used to
carry over enough metadata in order to convince
AWS iam that Bob acts with
Alice's approval. Let's see the
anatomy of a two p ticket.
So the part of the key is,
of course, sits under the admin,
meaning it's an admin only feature. Admin, two PA
ticket. And the value for that tag consists
of three areas. The first one denotes Alice's
identity, the third one, Bob's, and the middle can
have some sort of a payload which is kind of useful. We'll see later
on. Why. Now we're going to deep dive into the
control plane of the two PA SCP itself.
Hold your breath. The first thing first we
have to have some sort of an organizational chokehold
around who can even assign
source identities. Source identities in Aws are sticky
meanings. Once they're set on a role session, they cannot
be unset or even removed.
And here we request some sort of control
tag under meta, again, so not accessible
to admins, which denotes identity broker equals true.
And we're going to tag each and every one of the relevant OIDC
or Sam'l providers or even our homegrown identity
brokers with that role, which is going to guarantee
that if you attempt to set a source identity without that tag,
you are denied. And also,
we have a second statement here saying that you cannot do any
tagging operations on IAM if you don't have
a source identity attached, meaning you have to have a
source identity which has been dispensed to you by
an allow listed identity broker. The second part
deals with antireflexive integrity. What does
that mean? It means that the receiver section, the last
one, meaning that it can all
be set to the identity
of the principal making the request. Right? Meaning I
cannot give an approval ticket for myself.
I can give an approval ticket for any other source
identity in the system, whether it exists or not, but never to my
own. The third part deals with forgery,
meaning I cannot go about and forging a
ticket, say, to Bob, and telling it's
from Eve and not myself. This is
crucial because those typing operations are audited in
Cloudtrade, so we want to always have visibility into
who gave the ticket to whom.
In summary, it's an admin only feature.
It provides us with identity integrity,
anti reflexive integrity, anti forgery integrity,
and it is payload bearing,
meaning we can specify a CTO, a Unix timestamp
if you may, for that ticket, and have an external retention process
remove the tickets when the time is due.
And turns out it can't all be nice
and simple. IaM, we need to write some code for
that as well. So what we have here is the architecture for that.
It looks kind of complex, but it's fairly simple.
We have a lambda function that wakes up
every 30 minutes, scans the entire organization,
and just removes two PI tickets from each and
every role or user where they are
applied to, if their TTL has elapsed,
or if their TTL expires way
into the future, more than 4 hours into the future.
Right. Now that we got the control plane out of the way,
let's look at the data plane, and it's surprisingly short.
This is the entire data plane for that SCP.
Let's read it. It says you deny the actions
of deleting an s three bucket, an RDS instance or cluster,
unless you have an
approval ticket which is assigned for your own
source identity. And that's it.
That's the solution here. But then we looked at this
condition and we said,
hey, this looks very relevant. Let's copy and
paste that back into the control
tags SAP itself. So what does it
do in that context? In that context, what it
means is that you can escape the
guardrails of the control tags themselves by
having another admin's approval.
Meaning that if two admins collaborate, they can
tags or untag things outside the grant path.
So I'm not just bound to v one admin anymore,
I can tag v one, meta or v two hello world.
Whatever I like a second use case,
has more of a granular nature
to it. We said, great, we can restrict sensitive
actions, but now we want to restrict sensitive actions on some
resources. And we refer to this features as
two pa seals, which we also like to call two
PA seals. And the way it works is,
let's say I have an AWS secrets manager secret
and I want to tag it with seal equals total.
And once I do that, I want an admin room to
be able to describe that secret, but not to
reveal it, and never to remove the seal. Whereas the
same admin role with a two PA ticket can
describe the secret, reveal it, and even remove the seal.
So let's see how it's done.
But first, let's reflect about seals for a bit.
We're saying that sealing something totally,
it's useful for manually covered keys. That's how we store
vault keys in our organization. And sealing
trust relationship is kind of useful against privilege escalations.
And we'll see why in a bit. Let's explore trust relationship
seals. Right?
So the first part says that you
cannot bypass the seal without a two p a ticket. That shouldn't
be surprising, right? If the resource is sealed with a deny
trust update and you don't have a two PA ticket, you are unable
to update that roles.
Assume a policy. And the second part
here is the seal lifecycle. Right.
It means that you cannot change the seal or
delete it without a two PA ticket.
But press not, the end is near. Let's see just
how useful those trust relationship seals
are. This is what we call the two PA trust
relief pattern, and it gives us two
guarantees. The first one is you cannot
assume that role without an approval, without a
two PA ticket. And the second guarantee is that
you cannot make any changes to the trust policy without an approval.
That's a guarantee of the deny trust update two pa seal.
So here we have that trust policy that allows the
admin role to assume it, but denies any kind of action
if there is no two PA ticket attached.
And let's call that role super admin.
And at the bottom of the slide, you see that there's also
a tag on it, a two PA seal with a value of deny
trust update. Using that trust relay pattern
we just saw, we can extend the process of
approval to other systems like AWS eks.
The first thing we're going to do is make sure that the super
admin role is assigned to
systems masters on our EKs cluster, and that
the regular admin role is assigned to some sort
of auto group. And it's important to denote
here that this group does not have the equivalent of
IAM permissions within bks, right? So regular
admin cannot make new Kubernetes roles and Kubernetes
role bindings. The second part would be to provision
said super admin user with the trust policy
and the tags, the seal that we just saw and
that's it. That's how you relay the trust to an audience
systems, the same way that it has been
done for eks can be done for a hashicop
vault. Let's play a game of cat and mouse
for a second here. Let's suppose we have packed
admin in the system and that
packed is eyeing data within a sensitive bucket,
the system is protected. The bucket has
a two PA ticket God around it,
meaning the bucket policy cannot be
changed without a ticket. It also specifies that only the
sensitive role may access it. The sensitive role has a two PA
seal around it, and the super admin of
the system has a two PA seal
and requires a two PA ticket to assume.
That's a guarantee here. And the first thing the hacked admin
is going to attempt to do is get object or put bucket
policy, but both of those are denied by
the bucket policy, which kind of hardens the bucket.
Then the attacker may attempt CTO, assume the sensitive role,
or pass it on to Lambda or EC two machine where
it can privilege those permissions and gain access
to the bucket. But it is going to fail because it is denied
by the trust policy which only allows access from
the eks path. Then the actor
will attempt to change that trust policy and
it is denied by the two Paco because you must have an
approval ticket if you want to change the bucket policy. If you want
to change the sensitive roles policy, and then
the attacker might attempt to exact into
the EKs cluster he's going to try to edit
the AWS off config map which is in
the Kube system namespace, or even create role bindings or cluster
role bindings in order to elevate its own permissions.
But if we made sure that the regular admin role does
not inside the Kubernetes cluster does not have access
to those APIs, those will fail as
well. Then the hacker might attempt
to assume the super admin role and continue
from there to the eks cluster,
but it's unauthorized again because the
admin role requires a two k ticket in order to assume it.
The last thing an Acura might attempt to do is create his
own Sam'l provider or IDC provider
and then create two roles and have one
role attempt to sign an approval for the second role.
But that will fail as well because the sample
provider or IDC provider that the hackers just created does not
have the identity integrity clause assigned
to it. That identity broker equals true tag. So the two PA
SCP is going to deny that? Yeah, so it's all
control tags, always has been. We're using
it in production for more than two years
now. It's used by a team of more than ten
admins, and we're
using it for basically anything. It made the
process of creating new security features
on organization very agile and seamless because we
never have to stop and think about tagging integrity.
But thank you very much.