Disclosure: I work on Google Cloud.

Sorry about that. We (I) had screwed up and allowed users to smash themselves into the wall of "Huh, you keep getting preempted, but there's technically a slot right here". We've re-enabled some backpressure to keep you from getting hit over and over again, and instead see a stockout error when you should probably try another (less full) zone.

Hey, neat! Thanks for your work. I was hoping that this was the case, and I haven't tried since September. But I'll give it another shot now.

Feel free to send me an email (contact info in profile), if you want to go back and forth.

I killed a lot of time with this exact issue a couple months ago as well. Quite frustrating, and it was even worse trying to include the preemptible GPUs in a kubernetes cluster. It would often take so long to add the GPU to the cluster that it would get killed before it even ran any real workload. This cycle would repeat for an hour before a machine stayed up long enough to actually schedule kubernetes pods on them.

It would be nice of GCP didn't bill for pre-emptible instances that got system killed within 10 minutes of starting; or something reasonable like that.

I doubt it makes a material difference to their (or our) bottom line, but it's just a nice gesture and makes you never feel like "damn, I literally wasted money".

While this is pretty frustrating and seemed to be the result of a bug, I would personally only use preemptible instances are when I have everything scripted, and save the state of my work when you get a termination signal.

To me, they're not really sold as "cheaper, just not 100% guaranteed"; they're sold as "cheap, it will be killed at any moments notice so".

I haven't in awhile; when I used them more heavily, I got the full 24 hours. It's possible that it's region-dependent; IIRC I had better luck with us-east1 than us-central1.

I should test more at some point.