I don't think he personally designed the first implementation. But in any case, understanding of complex topics comes in waves.

I also think there’s this tendency in the industry to want to solve problems on your own without the help from outsiders, even if they know the problem space better than you do, and even if they’d gladly help (often for free) if asked. Sometimes it’s because you might not know whom to ask or recruit, but in this case, the vendor was known.

This is especially true for scaling. A solution that works great for your current deployment may be completely unworkable for 2x your current deployment.

You just won't know until you fall off the cliff. The armchair quarterback can opine that you should have just hired experts in XYZ domains from the start to design robust systems that can scale to arbitrary sizes, but most orgs don't need to scale to arbitrary sizes so this is highly likely to be wasted effort.

While I largely agree with you, this isn’t one of those cases. If Fly wasn’t supposed to scale in due course to this size, it probably wouldn’t have been funded. If your business model is predicated on you scaling, yes, you should hire appropriately in anticipation of that.

Besides, I’m not even necessarily talking about hiring here - even consulting would have been sufficient to avoid this catastrophe.

Yes, although it's rarely possible to know which bottlenecks will hurt the most up front. Unless you've done the same thing before, which is not the case with anyone pushing boundaries.

Basically this is an argument around so-called premature optimization. Good to have issues now while it is mostly enthusiasts that are the customers. Guessing that this bump will be forgotten in five years? And not like AWS et al don't have outages occasionally that they learn from.

Consul has been around for close to 9 years now, and people have in fact tried to use Consul in the very same way Fly did, in many different business and industries, with similarly failing outcomes. Hashicorp knows this and almost certainly would have counseled against it if asked.

Insert Donald Rumsfeld quote about un/known un/knowns.

This feels unnecessarily antagonistic. "If you were experienced, you would have made the right decision, _obviously_."

Did Fly.io kick your puppy or something?

I can see how it would be interpreted that way, and I apologize if it came across that way, but it wasn’t my intent. See my other comment below. What I’m really saying is that we need to be better about engaging subject matter experts early on when we are selecting technologies to power core business functions; and I think it’s a good illustration of why we need to continue to hire experienced people at startups.

That's a fair point... but at the same time, we shouldn't hold off on starting something just because we don't have perfect information.