News for hackers. Stuff that matters.
Re:Uber's technology is reportedly 'hanging by a thread' by imaginenore on Wednesday September 30, 2015 @07:51AM (#10303211)
From my perspective, Uber backend can be sharded trivially. No driver in LA is going to get matched with a passenger in NYC.
Re:Uber's technology is reportedly 'hanging by a thread' by charliesome on Wednesday September 30, 2015 @08:00AM (#10303260)
I generally find it wise to avoid calling engineering problems faced by other organisations trivial. There are often a lot of complicating factors that aren't obvious from an outside perspective.
Re:Uber's technology is reportedly 'hanging by a thread' by latch on Wednesday September 30, 2015 @08:23AM (#10303362)
Sure, but it's fun to talk about how you'd build it given some assumptions. I agree with your parent that it seems like a shardable dataset. To boot, I imagine that you could easily store all cars for a city/region in memory (type, x, y, driver id, status, ....).

Next I'd grid the area (maybe 250m2?) into blocks. When a user does a search, figure out his or her block, and start looking for cars in the current block, expanding outwards. You read-lock the blocks as you examine them. You only write lock 2 blocks as a car moves from one block to another (the blocks themselves would have a list of cars, maybe an array, maybe an rtree, maybe another grid).

It all falls apart when a single server can't handle all the load for a region. But then you could sub-divide the region, connect the servers with a queue (this car is now your responsibility), and let clients (not necessarily devices, but the api servers consuming these data servers) join the data.

Re:Uber's technology is reportedly 'hanging by a thread' by msandford on Wednesday September 30, 2015 @08:34AM (#10303408)
If you can have one server per city then I agree. Especially if those cities are small and you never travel between them. But places like LA totally blow that theory as you're going to have to have multiple servers or shards to cover everywhere.
Re:Uber's technology is reportedly 'hanging by a thread' by latch on Wednesday September 30, 2015 @08:44AM (#10303465)
Why LA, # of drivers, size of the city or # of users? Quick googling says LA is 500 miles squared, with Beijing being 6500.

The only case where I see 1 server failing is # of requests. All their drivers in the world probably fit in a few GB of memory (if that). 99% of a car's movement requires no locking (they stay in the same block), so there's very little write locking...I dunno...give it a 24 core server (or more)...

Maybe the problem is node and python. I don't know python runtime well enough, but this kind of setup is a nightmare for node. Sharing data across processes just isn't what it was meant to do (it rather fork and have a copy, but then your memory doubles per fork, and you have to keep it in sync). That's true for a lot of dynamic languages.

Re:Uber's technology is reportedly 'hanging by a thread' by msandford on Wednesday September 30, 2015 @09:19AM (#10303670)
Just using the google answer for "how big is LA" doesn't really cut it as there are millions of people that live outside of LA proper but still in the region that most people would call "LA". If you go to Wikipedia for Greater Los Angeles it's 34,000 square miles. https://en.wikipedia.org/wiki/Greater_Los_Angeles_Area

sqrt(34000) is nearly 200 miles on a side. And that doesn't really even cover all the areas that one might Uber from or to. So you'll have a lot of people crossing shard boundaries.

Greater LA has some 18mm residents, and again that doesn't count some of the places very close but not really IN the area. Places you might drive to in 15 minutes from the edge, over a pass.

Re:Uber's technology is reportedly 'hanging by a thread' by rco8786 on Wednesday September 30, 2015 @09:01AM (#10303554)
Thank you. I remember a year or so ago a similar article was written about Twitter and one of the HN armchair engineers argued that he could build Twitter on no more than 4 servers because it was so simple.
Re:Uber's technology is reportedly 'hanging by a thread' by jheriko on Wednesday September 30, 2015 @08:34AM (#10303409)
i dunno. i'd say they are dealing with a very classically well solved set of problems. its fairly trivial stuff... although i'm sure they have made their own problems for themselves whilst approaching it.
Re:Uber's technology is reportedly 'hanging by a thread' by matthewrudy on Wednesday September 30, 2015 @08:32AM (#10303400)
Looking at Uber's APIs they're built to be sharded.

But sharding comes at the cost of complexity.

I work on an uber-like service and we use crude sharding between countries.

The problem is:

* sometimes your sharding is too coarse - eg. two cities which have no cross traffic

* sometimes your sharding is too fine - eg. cross border traffic

But the real complexity is;

* managing configuration and logic differences between countries and cities

* hosting so many different sharded clusters, routing between them, and keeping them all updated.

And that's where all the work goes.

Re:Uber's technology is reportedly 'hanging by a thread' by mwhuber on Wednesday September 30, 2015 @08:58AM (#10303542)
One of Uber's architects talking about how they're handling that over via InfoQ

http://goo.gl/bAoFHi