From what I gathered off of several HN comments of Uber insiders, the size of their engineering team is actually more likely to be one of the causes for bad engineering decisions. They seem to have too many engineers running around for too little actual work to be done, which results in those engineers coming up with stuff to do to keep them busy (and of course to ensure they appear to be busy and worth their money to their superiors).

One of the best ways to create yourself some work is to not choose an already-proven path to a solution, but invent a new one just for the sake of inventing a new one. Of course that's not how this kind of doing is justified - the justification is usually "the proven path does not scale to our needs" or "by using a special approach adapted to our needs we can be more efficient" or "the proven path is too complex, we can get by with something simpler and easier to maintain". Which might actually all be proper justifications, it's just that you should have some hard proof for these statements, like benchmark results of a comparison of different approaches. That part often gets skipped, which is actually ironic, because doing extensive evaluation and benchmarking and implementing different approaches first before choosing one for production actually serves quite well to create even more work to do.

Also, it's resume-driven development. Five years later they want to be able to be the guy who can say, "Yeah, I was the original designer behind Kafka," but substituting in a brand-new Uber technology.

It might be sub-optimal for Uber but it's better for the industry as a whole to have many different implementations of the same type of solution.

And none of their blog posts about their own geo stuff ever come close to answer the why. What is wrong with e.g. PostGIS or Elasticsearch? Why do they need to reinvent geo databases several times?

Teams outside Infra aren't allowed persistent disk. Service authors get to pick from a small catalog of managed operational storage systems. Life would be a lot easier if a scalable Postgres were one of them, but it isn't. For the moment, the offerings with high scalability and good SLAs are all KV stores.

We do have ES but it's operated for log search, not production critical paths.

Too much time and not enough things to do.

How does this happen?! They're so expensive

Give me twenty billion dollars to make a ride hailing app and I'll show you.

More seriously: they raised tens of billions of dollars to make a ride hailing app and research self-driving cars. Throwing more money at a problem doesn't solve it faster, but it does pay for hiring people, and headcount is seen as a proxy for doing stuff.

When a measure becomes a target, it ceases to be a good measure.

https://en.wikipedia.org/wiki/Goodhart%27s_law

Some big start-ups[0] solve this conundrum by investing in adjacent companies to find the solutions they're paid to find. Outsourcing! This still has limits because there's so much money flowing around right now. Tossing a few million at a company with hundreds still won't solve the problem faster.

[0] Start-up definition for this post: a company that has taken funding but hasn't yet found a sustainable business model. That's how Uber is still a start-up with more money than most companies make in decades.