That's the part that surprised me as well; it doesn't seem like a field that should be eligible for anything other than an exact match. I am unable to conceive of a use case for pattern matching account IDs.

If for some reason you’re dealing with thousands of accounts that are architecturally indistinguishable, bucketing them by ID prefix isn’t a particularly wild thing to want to do.

AWS assigns these individually, and customers can’t influence the ID that they get. For access control purposes I see no valid use case for wildcards there.

Sharding on account ID might make sense if someone has a large number of them, but that would not necessitate wildcard matching.

It sure could. If you're sharding by id and have some per-shard resources, they could definitely get permissions to only accounts 12345*. (I'm not saying it's a good idea, just that once you're in that situation, you would pattern match on partial IDs)

But it could seem like a neat and obvious way to reduce policy size (which is limited) and make it arguably more readable, or at least the intention clearer. (I might assume `2847373847261`, `37385857721`, `5847262671`, ... is `*1` over our accounts, but I might be wrong, or I might forget (/not correctly automate) to add the new one.)

But account IDs are assigned by Amazon and there's no structure within the namespace that's useful to you. If you mean all, you can wildcard * - but there doesn't seem to be any legitimate cases for "all account ids beginning with a 1".

Presumably the permissions language is broadly defined and has something like

    filter = property, operand, value

with few constraints on which operands can be used in which situations, to keep parsing the language simple (parsers being notoriously prone to vulnerabilities after all). In retrospect perhaps that isn't a good trade-off, but it would be tricky to tighten things up now without breaking lots of existing users.

If there are a lot of them and there is a b-tree index somewhere you might find it useful to scan them in alphabetical order.

At that point you should be using AWS Organizations and OUs.

I think the GP is talking about granting access to a particular bucket to an unbounded number of customer AWS accounts — probably in requester-pays config. (Think: static data used by an Amazon Marketplace virtual appliance.)

Those wouldn't follow any particular partial pattern though would they?

They don't need to — in fact, you might be depending on them not following any particular pattern. Think: treating the Account IDs as pre-hashed keys, and then specifying prefix patterns as ways of sharding the hash keys onto a set of buckets, to evenly distribute access (and therefore traffic) by customer.

Do you ever really need to do that on AWS though? Can you really bog down S3 by having loads of requests to a single bucket?

GET requests? Probably not. PUT/DELETE requests? I think so, yes. All updates to the bucket ultimately bottleneck at an update to the meta-version of the bucket-record-object in the object-storage-system's bucket-metadata store (itself probably something like DynamoDB / BigTable / etc.)

Given the way these IaaSs' distributed KV stores all manage writes (i.e. by having a cluster of transactor nodes that per-key write-linearization responsibility for parts of the keyspace is sharded across — such that writes are fanned out to a particular designated transactor-node given the key's hash-slot), a very large S3 user, generating an extremely high level of metadata-update concurrency against a bucket, could very likely write-contend that bucket's metadata / have a "hot" bucket-metadata key; experience low perf due to that; and solve that by sharding the bucket (swapping one too-hot metadata key for N somewhat-hot metadata keys.)

I want to give an intuition-building example here, of an IaaS feature that wouldn't exist / wouldn't be exposed to the user if not for object-storage buckets being metadata-write-contended at scale. I'm not very familiar with the AWS ecosystem, though, so I'm not sure what the good example is for AWS. What I do know is GCP, so here's a GCP example: Google Cloud Dataflow allows you to set a temporary workspace GCS bucket on a per-job basis (gcsTempLocation). And, IIRC, Google's Cloud Architects advise to not have a bunch of active Dataflow jobs sharing the same gcsTempLocation — regardless of whether they use distinct key prefixes to namespace the temp files. Given that each job would be doing a lot of little serial updates to the temp bucket — and given that Dataflow jobs can each be highly internally concurrent — you're already potentially putting out O(N^2) ~concurrent updates to that bucket. You really don't want to make it O(N^3).

Can you even block off IDs like this?

If you’re big enough to need it I’m sure it can be arranged.

If you're big enough to think you need it, you're also big enough to have people who can tell you it's a bad idea and there's a better tool for the job.

aws support is not going to bend over backwards just to let you shoot yourself in the foot. it's more likely they grant an exception to one of the iam quotas.

Bucketing them by prefix of the end-user ID is not exactly smart.

Either you bucket by an internal ID and give the user a hash, or you give the user an ID and bucket by your internal hash.

Users have no business knowing your sharding scheme.

Well, that assumes that the ID is cryptographically random. Perhaps that is a bad assumption.

My general assumption is not that they’re random, but at least that they’re not correlated; in particular that Amazon is not in the habit of handing out, like, account IDs 676363687000 - 676363687999 to a single organization. Even if they did hand out a sequential batch of 1000 account IDs, it would be more likely to be 676363687541 - 676363688540 than a set with a single consistent prefix.

Odds are that an account wildcard match like 676363687* will just match a few hundred entirely random AWS accounts.

> in particular that Amazon is not in the habit of handing out, like, account IDs 676363687000 - 676363687999 to a single organization

Honestly, wouldn't surprise me that much if they were willing to accommodate this if for sufficiently large accounts. It'd still pretty sketchy to design your access control around, but it wouldn't be unrealistic.

I once was involved in creating two (linked) amazon accounts at the "same" time, and ended up with account IDs of which the first 4 digits are identical.

A namespace you and only 100 million other accounts share - probably reasonable to just grant access to “1234*”.

Even if it's not that doesn't mean pattern matching IDs is good.

Oh god, no. Seems like a bad idea indeed! But might give some insight into their system.

If it's sequential, that somehow seems worse?

it's irrelevant whether they're "cryptographically" random, all that matters is that account IDs are not controlled by the user and therefore have no logical relation to any access-control policies the user may wish to implement

This is because, I assume, the AWS policy execution has a number of “operators” and “operands”, and in this case, you’re using the StringLike operand on the account ID string.

Anyway, this discussion is a bit amusing to me, since Devops people are discovering side channels[1] now, although other types of side channels such as speculative execution side channels on CPUs (Meltdown, Spectre) already made waves at the time of discovery, and before that we had power analysis[2] and Magnetostriction detection, and constant time cryptography (this one is a field of its own, so I omit references.)

[1] https://en.m.wikipedia.org/wiki/Side-channel_attack [2] https://en.m.wikipedia.org/wiki/Power_analysis

> the AWS policy execution has a number of “operators” and “operands”

That is correct.

The IAM condition language is flexible and does not prevent you from doing strange things.

It’s a consequence of weak typing choices - not an inevitable result of allowing flexibility.

Doing glob matching on account IDs is like doing concatenation with guids, applying a bitshift to a UTF8 string, or running a regex on an integer. It is a nonsensical operation, and - as shown here - results in surprising security properties of the resulting system.

Surprising security properties are an undesirable result in an access control policy language.

Of course it's possible to create a more sophisticated system with less possibility for user error.

They have ARN's which include the account which a glob match is useful. Something like "arn:aws:*:*:1234567890:*" is useful but "arn:aws:*:*:1234567*:*" isn't

> Anyway, this discussion is a bit amusing to me, since Devops people are discovering side channels[1]

We, the DevOps people, already knew about side channel attacks, Spectre and the likes, evaluated the performances hit for the fixes (or alleged fixes), patched our kernel boot params etc etc. We are curious people, just like many here.

Jesus, I have been in tech for 25 years and we're still throwing shit across this divide.

Well we are, if nothing, consistent.

The next nerd who pinches and inch of the coder/ops divide is going to make a billion.

Figma did this for design-dev and is worth about $40 billion.

It's amazing how anything got made considering how disjointed processes used to be.

Only worth about 10 billion. But still a lot!

The deal was for $20b my mistake.

The Adobe deal didn't go through. Last publicly hinted valuation was back at ~$10B when employee share buyouts happened in JAN 2024.

¯\_(ツ)_/¯

Not GP, I tried to answer as friendly as possible. Their approach is even more weird given that Security goes really hand in hand with systems infrastructure In any small enough organization, security will be managed by devops/sre. If the org becomes large enough, it will start dedicating people to Security.

I didn't think you were wrong I just thought the exchange and its tone was very stereotypical of Developers and Operations (and I was amused). Each being on their side of the fence and only throwing things over the top.

It used to be really bad... "works on my machine", bad deploy instructions etc. We paved a lot of these roads with (really fucking bad) strategies like containers. At some point maybe we will learn to write software that is operationalizable. Projects like tigerbeetle give me hope that this is our next evolution.

Probably comes out of a tendency to generalize. Last week for one of my side projects I built something that lets you write queries in a format inspired by OWL and it has a library of relational operators that can, say, extract the host from a URL or do a prefix query, like query, regex query, etc.

As it is my side project’s side project I do what is easy so these operators are always available even in cases where they don’t make sense (I dunno what happens if you try a regex query on a number, I don’t care) I can imagine there is something a bit like this inside AWS but for a security-sensitive system with a lot of users you have a different standard.

This is like matching a bitfield on the group ID on a Unix system. I could see someone coming up with this idea, and I could see them thinking this is somehow smart, but implementing it on systems that aren't 100% in your control would just be silly.

> there is no legitimate reason why you would grant or deny permission based on a partial account id match.

You're not a fan of my AWS lottery idea where accounts ending in 666 get access to a free bitcoin miner??

I love the idea, but maybe we can pair it with my ICO idea of flipping a virtual stack of Pogs to unlock new units of currency that can be mined.