News for hackers. Stuff that matters.
Re:What I Wish I Had Known Before Scaling Uber [video] by andy_ppp on Wednesday September 28, 2016 @10:14AM (#12597745)
I always wonder in these cases about giving each car an actor in Erlang/Elixir and having complete network transparency, message handling and crashes handled for free.

The routing is very complex too but as you note scales well, until you want to start routing/pickups based on the realtime location of other cars.

Re:What I Wish I Had Known Before Scaling Uber [video] by ramchip on Wednesday September 28, 2016 @11:13AM (#12598266)
I've given that kind of model a lot of thought too, but I did not find very satisfying solutions to these problems:

- How do you distribute the car actors on nodes, assuming the number of nodes is variable? (I think riak_core looks interesting, but it does not seem to have a way to guarantee unicity of something - it's rather built to replicate the data on multiple nodes for redundancy)

- What happens if a node fails? What mechanism is going to respawn the car actors on a different node? How do you ensure minimal downtime for the cars involved?

- What happens if there's a netsplit, e.g. how do you ensure no split brain where two nodes think they're responsible for a car?

It feels to me like the traditional erlang-process-per-request architecture coupled with a distributed store (riak or w/e) makes it possible to avoid the very difficult problem of ensuring one and only one actor per car in a distributed environment.

Re:What I Wish I Had Known Before Scaling Uber [video] by andy_ppp on Wednesday September 28, 2016 @06:11PM (#12601759)
For the netspit stuff I'd look at Phoenix.Presence works and see how that handles it with a CRDT. There are various ways of distributing things across nodes in Erlang/Elixir but maybe you'd need to build something.

I think you are right about the riak core stuff - you could probably keep track of cars using some sort of distributed hash and kill multiple cars if they were to ever spawn.

In fact a way of instantiating processes and finding them based on a CRDT is probably a pretty cool little project...

Re:What I Wish I Had Known Before Scaling Uber [video] by ramchip on Wednesday September 28, 2016 @07:39PM (#12602226)
I think that's a cool idea. It would have the downside that error recovery could take a while though, depending on the permdown period, so during that time a driver would be stuck; while in a request-based system they can immediately try again and it would work (hit a different instance). But it might not be too bad.
Re:What I Wish I Had Known Before Scaling Uber [video] by andy_ppp on Sunday October 09, 2016 @03:28AM (#12670231)
I've had uber crash on drivers while on a ride, even 30 seconds is better than complete failure! I think it would make uber more resilient...
Re:What I Wish I Had Known Before Scaling Uber [video] by erichocean on Wednesday September 28, 2016 @01:24PM (#12599510)
> having complete network transparency, message handling and crashes handled for free

That describes no system, ever, including Erlang. TANSTAFL

Re:What I Wish I Had Known Before Scaling Uber [video] by andy_ppp on Wednesday September 28, 2016 @06:26PM (#12601851)
Care to explain which bits Erlang/Elixir doesn't do or at least provide a solution for? As the sibling show you may need to be smart about certain things but the OTP definitely provides ways of solving the above things.
Re:What I Wish I Had Known Before Scaling Uber [video] by lgas on Wednesday September 28, 2016 @10:50AM (#12598069)
You're saying you want to extend "just let it crash" to the individual car level? :)
Re:What I Wish I Had Known Before Scaling Uber [video] by andy_ppp on Wednesday September 28, 2016 @06:22PM (#12601819)
Depends on how smart your supervisor gets doesn't it but sure, why not? Many times processes can be restarted in microseconds...