News for hackers. Stuff that matters.
Re:AWS vs. GCP reliability is wildly different by sitharus on Wednesday September 21, 2022 @07:49PM (#32933157)
This is a very good point - AWS uses tags to give instances a friendly name, so the name does not have to be unique. The same logic would not fail on AWS.
Re:AWS vs. GCP reliability is wildly different by smashed on Thursday September 22, 2022 @12:29AM (#32934804)
Which makes 2000% sense.

Why would any tenant supplied data affect anything whatsoever?

As a tenant, unless you are clashing with another resource under your own name, I don't see the point of failing.

aws S3 would be an exception, where they make that limitation on globally unique bucket name very clear.

Re:AWS vs. GCP reliability is wildly different by jsolson on Thursday September 22, 2022 @01:28AM (#32935089)
Idempotency.

You're inserting a VM with a specific name. If you try to create the same resource twice, the GCE control plane reports that as a conflict.

What they're doing here would be roughly equivalent to supplying the time to the AWS RunInstances API as an idempotency token.

(I work on GCE, and asked an industry friend at AWS about how they guarantee idempotency for RunInstances).

Re:AWS vs. GCP reliability is wildly different by stevenguh on Thursday September 22, 2022 @02:04AM (#32935240)
GCP control plane is generally not idempotent.

When trying to create the same resource twice, all request should report the same status instead one failing, one succeeding.

In AWS, their APIs allow you to supply a client token if the API is not idempotent by default.

See https://docs.aws.amazon.com/AWSEC2/latest/APIReference/Run_I....

Re:AWS vs. GCP reliability is wildly different by jsolson on Thursday September 22, 2022 @02:23AM (#32935354)
> When try to create the same resource twice, the second should report success instead of failing.

Before I quibble with the idempotency point: I agree with this, entirely, but it is what it is and a lot of software has been written against the current behavior. So I'll cite Hyrum's law here: https://www.hyrumslaw.com/

> GCP control plane is generally not idempotent.

The GCE API occupies an odd space here, imo. The resource being created is, in practice, an operation to cause the named VM to exist. The operation has its own name, but the name of the VM in the insert operation is the name of the ultimate resource.

Net, the API is idempotent at a macro level in terms of the end-to-end creation or deletion of uniquely named resources. Which is a long winded way of saying that you're right, but that from a practical perspective it accomplishes enough of the goals of a truly idempotent API to be _useful_ for avoiding the same things that the AWS mechanism avoids: creation of unexpected duplicate VMs.

The more "modern" way to do this would be to have a truly idempotent description of the target state of the actual resource with a separate resource for the current live state, but we live with the sum of our past choices.

Re:AWS vs. GCP reliability is wildly different by Terretta on Thursday September 22, 2022 @06:45AM (#32937123)
It's a lot shorter to write:

You're right, we did it wrong.

// And paradoxically makes engineers like you.

Re:AWS vs. GCP reliability is wildly different by jsolson on Thursday September 22, 2022 @11:04AM (#32939851)
Sure, except I think that at a macro level we got it more right than AWS, despite some choices that I believe we'd make differently today.
Re:AWS vs. GCP reliability is wildly different by akramer on Thursday September 22, 2022 @11:24AM (#32940178)
The GCE API can be idempotent if you'd like. Fill out the requestId field with the same UUID in multiple instances.insert calls (or other mutation calls) and you will receive the same operation Id back in response.

Disclaimer: I work on GCE.

Re:AWS vs. GCP reliability is wildly different by jsolson on Thursday September 22, 2022 @01:28PM (#32942029)
Today I learned! I'll admit I didn't know this functionality existed, and I've instead had used instances.insert following by querying the VM resource.

This is nicer!

Re:AWS vs. GCP reliability is wildly different by bushbaba on Thursday September 22, 2022 @01:34AM (#32935115)
Do you really need idempotency for runVM though.
Re:AWS vs. GCP reliability is wildly different by jsolson on Thursday September 22, 2022 @02:09AM (#32935272)
I mean, it's kinda nice to know that if you reissue a request for an instance that could costs thousands of dollars per month due to a network glitch that you won't accidentally create two of them?

More practically, though, the instance name here is literally the name of the instance as it appears in the RESTful URL used for future queries about it. The 409 here is rejecting an attempt to create the same explicitly named resource twice.

Re:AWS vs. GCP reliability is wildly different by philliphaydon on Thursday September 22, 2022 @06:57AM (#32937203)
Sounds like AWS got it right.
Re:AWS vs. GCP reliability is wildly different by jsolson on Thursday September 22, 2022 @11:01AM (#32939796)
You're entitled to that takeaway, but I disagree. I believe GCP's tendency to use caller-supplied names for resources is one of the single best features of the platform, particularly when compared against AWS's random hex identifiers.

Note that whether this creates collisions is entirely under the customer's control. There's no requirement for global uniqueness, just a requirement that you not try to create two VMs with the same name in the same project in the same zone.

Re:AWS vs. GCP reliability is wildly different by philliphaydon on Thursday September 22, 2022 @12:21PM (#32941101)
With GCE can you create 10 instances or do you need to create all 10 individually?
Re:AWS vs. GCP reliability is wildly different by jsolson on Thursday September 22, 2022 @01:19PM (#32941895)
As far as I know, the `instances.insert` API only allows individual VMs, although the CLI can issue a bulk set of API calls[0], and MIGs (see below) allow you to request many identical VMs with a single API call if that's for some reason important.

You can also batch API calls[1], which also gives you a response for each VM in the batch while allowing for a single HTTP request/response.

That said, if you want to create a set of effectively identical VMs all matching a template (i.e., cattle not pets), though, or you want to issue a single API call, we'd generally point you to managed instance groups[2] (which can be manually or automatically scaled up or down) wherein you supply an instance template and an instance count. The MIG is named (like nearly all GCP resources), as are the instances, with a name derived from the MIG name. After creation you can also have the group abandon the instances and then delete the group if you really wanted a bunch of unmanaged VMs created through a single API call, although I'll admit I can't think of a use-case for this (the abandon API is generally intended for pulling VMs out of a group for debugging purposes or similar).

For cases where for whatever reason you don't want a MIG (e.g., because your VMs don't share a common template). You can still group those together for monitoring purposes[3], although it's an after-creation operation.

The MIG approach sets a _goal_ for the instance count and will attempt to achieve (and maintain) that goal even in the face of limited machine stock, hardware failures, etc. The top-level API will reject (stock-out) in the event that we're out of capacity, or in the batch/bulk case will start rejecting once we run out of capacity. I don't know how AWS's RunInstances behaves if it can only partially fulfill a request in a given zone.

[0]: https://cloud.google.com/compute/docs/instances/multiple/cre...

[1]: https://cloud.google.com/compute/docs/api/how-tos/batch

[2]: https://cloud.google.com/compute/docs/instance-groups

[3]: https://cloud.google.com/compute/docs/instance-groups/creati...

Re:AWS vs. GCP reliability is wildly different by vineyardmike on Friday September 23, 2022 @04:23AM (#32949317)
> unless you are clashing with another resource under your own name, I don't see the point of failing.

Is that not the conclusion? The tester was clashing with their own names?