News for hackers. Stuff that matters.
Re:The Uber Engineering Tech Stack, Part I: The Foundation by Animats on Sunday July 24, 2016 @03:18AM (#12152583)
I have to admit, I could see this running for a city the size of SF on a desktop machine under the table at the taxi depot. Uber has 11,000 drivers in SF, but probably only a few thousand are on at any one time. A ride takes a few minutes, so if you figure 3,000 active drivers and 4 rides per hour, that's only about 3 ride transactions per second. You have a transaction at ordering, one at ride start, and one at ride end. Plus you have tracking of where all the active drivers are, pinging maybe once a minute. That adds up to only at 10-20TPS. You can offload the routine web and app stuff to some front end machines. And you want to do some analysis every minute or two to see where there are "surges".

The only non-trivial part of this is assigning drivers to rides.

Re:The Uber Engineering Tech Stack, Part I: The Foundation by ricardobeat on Sunday July 24, 2016 @05:57AM (#12152867)
It's easy to imagine the simplest stack that can serve the core features of any service, and that is well served by a single box. What's missing from the picture is the infrastructure to replicate this 500 times by separate teams, monitoring all of it, backup, auditing, aggregating customer and business metrics, back-office systems, and more. Plus the fact that these things always grow organically and embed a host of imperfect decisions - the imaginary system will always be better designed.
Re:The Uber Engineering Tech Stack, Part I: The Foundation by tlrobinson on Sunday July 24, 2016 @06:57AM (#12152987)
This. Armchair software architecture is so easy when you can gloss over the details that make a product great.

Also routing, ETAs, geocoding/search, etc.

WebGL visualizations and such are probably overkill, but if it makes the company more fun to work for then it probably breaks even, at worst.

Re:The Uber Engineering Tech Stack, Part I: The Foundation by danpalmer on Sunday July 24, 2016 @09:40AM (#12153266)
Couldn't agree more. It's all the invisible details that cause the load.

It's a much more trivial example, but highlights the point well I think - we have pages in the app I work on that would respond in ~100ms, but might have a single sentence on them that takes another 100ms to generate because of the complex data relationships involved in figuring out what that sentence needs to say. The 'request handler' might be 20 lines of code, with a 50 line util function to generate that line of text. No armchair architect will ever take into account things like that, but the end result is a page that is just a bit more personalised to the user and therefore improves their experience.

In an app of any real size, I imagine there are anywhere from hundreds to many thousands of tiny little details like this that all together drastically increase the amount of power needed to run a service.

Re:The Uber Engineering Tech Stack, Part I: The Foundation by orf on Sunday July 24, 2016 @09:58AM (#12153313)
> No armchair architect will ever take into account things like that

An armchair architect would say it's not needed. They would question whether spending 50% of your response time generating a single sentence is in any way worth it, and wonder what kind of architectural mistakes led to that.

Re:The Uber Engineering Tech Stack, Part I: The Foundation by jdavis703 on Sunday July 24, 2016 @11:27AM (#12153535)
The problem with this line of reasoning is that it implies the business exists to serve the software. Unless you work at a tech-focused non-profit, the software actually exists to serve the business.
Re:The Uber Engineering Tech Stack, Part I: The Foundation by orf on Sunday July 24, 2016 @03:32PM (#12154469)
> the software actually exists to serve the business.

Sure it does, but the business also wouldn't exist without the tech in Ubers case (and a lot of other cases). And it's going to be your head on the line when you keep adding these 100ms sentences because the business wants it for no good reason and your page takes 3 seconds to load, and nobody buys anything from the site.

Re:The Uber Engineering Tech Stack, Part I: The Foundation by robbles on Sunday July 24, 2016 @03:52PM (#12154551)
You're making the assumption that the additional features slowing down the service aren't adding value.

More common is a "death by 1000 cuts" scenario where the various causes of slowness are apparent to the developers, but quite difficult to remove because they've become necessary to the continued success of the business.

Re:The Uber Engineering Tech Stack, Part I: The Foundation by orf on Sunday July 24, 2016 @05:42PM (#12155047)
> You're making the assumption that the additional features slowing down the service aren't adding value.

No, I'm questioning whether the value added is greater than the value lost, and in this hypothetical example clearly not. So it's your job to point that out to whoever and not silently obey.

Re:The Uber Engineering Tech Stack, Part I: The Foundation by Animats on Sunday July 24, 2016 @04:09PM (#12154631)
The Uber attract screen that seems to show lots of cars near your location has no relationship to reality.

WebGL is mostly client side. Ship the list of data points and let the client do it.

Re:The Uber Engineering Tech Stack, Part I: The Foundation by gaius on Sunday July 24, 2016 @11:13AM (#12153500)
We saw this in the 90s, people with a whole rack of machines running Java or Perl CGIs to serve a site with less traffic than we were doing with a single, ordinary box running NSAPI. You need loads of scaffolding that you mention, only if you are trying to fit a square peg into a round hole.
Re:The Uber Engineering Tech Stack, Part I: The Foundation by mkagenius on Sunday July 24, 2016 @03:50AM (#12152645)
> tracking of where all the active drivers are, pinging maybe once a minute.

Once every 5 seconds will be more accurate.

Re:The Uber Engineering Tech Stack, Part I: The Foundation by dzhiurgis on Sunday July 24, 2016 @12:21PM (#12153690)
Driver app pings central server every 15 seconds. If any ride requests are available, it displays it to driver.
Re:The Uber Engineering Tech Stack, Part I: The Foundation by ambicapter on Sunday July 24, 2016 @10:10AM (#12153341)
And will 30x the number of requests...
Re:The Uber Engineering Tech Stack, Part I: The Foundation by karmelapple on Sunday July 24, 2016 @08:30AM (#12153140)
I'm wondering why matching is nontrivial.

When I use these services, it doesn't always give me the closest car of the ones shown. It's all about which driver accepted my request first. Isn't it a fairly easy calculation of "all cars within X km or the nearest Y cars", and then whichever driver taps first gets matched with you?

Re:The Uber Engineering Tech Stack, Part I: The Foundation by dkarapetyan on Sunday July 24, 2016 @03:26AM (#12152606)
Indeed. Most SaaS operations are over-provisioned both in terms of hardware and software.
Re:The Uber Engineering Tech Stack, Part I: The Foundation by goldenkey on Sunday July 24, 2016 @09:39AM (#12153262)
And over-provisioned in terms of employees, I'd add.