News for hackers. Stuff that matters.
Re:A former Uber engineer's disaster story by dreamcompiler on Thursday December 10, 2020 @12:25PM (#25375998)
Follow-on tweets address this somewhat: Every airport has different pickup locations, every region has different payment rules including cash, etc.

Apparently all this location-specific business logic has to be in the app itself and is not downloadable on demand.

Re:A former Uber engineer's disaster story by oefrha on Thursday December 10, 2020 @12:55PM (#25376357)
Yet, the last time I was throttled to 2G speed due to a problem with my carrier, the Uber app couldn’t even start (getting to the first interactive screen) in like five minutes. What’s the point of distributing the entire world’s data, which gotta change from time to time, when you can download the tiny requisite piece each time, especially considering that a relatively high speed Internet connection is required to use the app anyway?

Edit: https://news.ycombinator.com/item?id=25376346 goes into more details.

Re:A former Uber engineer's disaster story by lhorie on Thursday December 10, 2020 @01:51PM (#25376975)
Uber actually has a not-so-well-known product for this very problem: the mobile web version (m.uber.com)

It doesn't have nearly as much functionality as gregdoesit mentioned, but it's enough to service the use case of getting you from point A to point B when you're stuck w/ crappy connectivity.

Re:A former Uber engineer's disaster story by kingnothing on Thursday December 10, 2020 @02:03PM (#25377121)
The other not-so-well-known product is the Uber Lite app (iOS and Android) which was specifically made by Uber for this problem, particularly in the developing world.
Re:A former Uber engineer's disaster story by sethammons on Thursday December 10, 2020 @01:55PM (#25377025)
Wow, I had no idea! On app load, maybe checking speed and then displaying a link to the web version for low bandwidth areas would be a good move.
Re:A former Uber engineer's disaster story by saagarjha on Thursday December 10, 2020 @02:01PM (#25377099)
Why display a link? Hybrid web/native apps are a tried-and-true strategy when you have massive complexity or dynamic content that you don't want to ship with your app. If you use it judiciously and tastefully the "native UI everything" people don't notice and come after you, either.
Re:A former Uber engineer's disaster story by nudgeee on Thursday December 10, 2020 @06:19PM (#25380133)
Fun fact: if you open the Uber app and hit the Food icon/tab, the experience is entirely a web one: it is powered by ubereats.com in a webview.
Re:A former Uber engineer's disaster story by sethammons on Thursday December 10, 2020 @02:37PM (#25377570)
Not an app developer ;). But this makes sense. Thanks
Re:A former Uber engineer's disaster story by wwweston on Thursday December 10, 2020 @03:27PM (#25378157)
Wish they still had what they did when I started using it: send an SMS to a shortcode number with your pickup location.
Re:A former Uber engineer's disaster story by ztravis on Thursday December 10, 2020 @09:50PM (#25381701)
I actually built this product out (for both Uber and Lyft) - SMS/MMS-based ride requests (MMS only for sending google maps PNGs to confirm pickup location). I was about to convert it from a personal project (I used it to get rides on both services for a few years) to a "real" one open to customers, etc and then ran into snags with both APIs at around the same time. Uber's API changed to potentially require a web flow to confirm surge pricing (IIRC - now it's scheduled to be disabled entirely very soon) and Lyft's API was basically shut off entirely (there's a way to apply but I was unable to make any progress even through internal contacts at Lyft). I'll try and keep an eye out for any changes...
Re:A former Uber engineer's disaster story by lock-free on Thursday December 10, 2020 @12:44PM (#25376250)
There are a lot of ways to frame these problems as interesting algorithms but it doesn't really strike me as something that is troubled by scale.

I've worked on similar problems where "everything is a special case" in some vein and it takes a few very clever minds to construct the algorithms and datastructures that can come to a good solution. But that's just it, it's the work of a few engineers and not thousands writing if-else blocks for each airport.

I still don't understand what Uber does that requires so many people getting paid so much money, the only thing that makes sense to me is they don't want them working for Lyft. It would be helpful for people to shed more light on how engineering resources are spent and what particular systems require so many hands to realize, but that's too valuable information to leak I'd imagine.

Re:A former Uber engineer's disaster story by dntrkv on Thursday December 10, 2020 @01:08PM (#25376501)
A good place to start would be to go through the Uber UI and click through every button and every page that exists.

Now create a driver profile and do the same thing.

Now create the multitude of other special profiles they have and do the same.

Now do the same for Uber Eats.

Now do the same for Eats but using the restaurants version of the app.

And the couriers.

Now get in on Uber Freight (and any other projects they have) and do all of the same.

Now do all of that again for their other platforms (iOS, Watch OS, Wear OS, Android, Web)

Now access their internal tools that make all of this possible and do the same.

Now think about localizing all of that. Handling special cases for different places/languages/legalities. Think of all the different payment integrations. Security issues. Infrastructure. Technical debt. AB testing. Metrics. Dev tooling.

And I've only begun to scratch the surface here. The point is there is a lot more than meets the eye, especially when operating on a global scale.

Re:A former Uber engineer's disaster story by saagarjha on Thursday December 10, 2020 @01:27PM (#25376706)
This is a common but specious argument. It is possible to make large, complicated apps without creating a monstrosity. That you are making something like this is a manifestation of an engineering culture that has too many cooks in the kitchen, a lack of specific engineering talent spent on finding issues and telling the teams shipping them to stop, people shipping libraries that they don't need and tech debt that was never pruned away. An app with 100x more people working on it should not be 100x larger, because it certainly won't have 100x more features in it. Sure, Uber is more than "a map and a couple views" but the fact remains that their app should never have been in the state that it was.
Re:A former Uber engineer's disaster story by wsinks on Thursday December 10, 2020 @01:45PM (#25376909)
For my own curiosity, what large, complicated apps are not monstrosities in your eyes?
Re:A former Uber engineer's disaster story by saagarjha on Thursday December 10, 2020 @01:59PM (#25377072)
The Apple Maps app on iPhone, for example? If you make it fair and count the code that goes into the frameworks specifically for it, it's still tens of megabytes at the most. There is a huge part of it running server-side to support it, of course, but it is difficult to say that it is not a complicated app. Or consider the Mail app? It needs to deal with IMAP, and has custom flows for a number of named mail services (Gmail, Yahoo, etc.), and then it has to have special code to handle all the edge cases of "what happens if the user deletes a message here but it didn't sync, or Google sends us 503s sometimes if we do this, or…" You can make arguments that maybe it is slightly less or more complex than Uber, but it is just a couple dozen megabytes and not hundreds.
Re:A former Uber engineer's disaster story by jimmydorry on Friday December 11, 2020 @09:39AM (#25386275)
10s of megabytes? Where did you pull that number from? Since it is a system app, its true install size is hidden away from users. The comparable google maps app is 190 MB.
Re:A former Uber engineer's disaster story by saagarjha on Friday December 11, 2020 @01:19PM (#25388827)
I counted up the system frameworks it relies on that can be reasonably considered to exist for the purposes of serving the app.
Re:A former Uber engineer's disaster story by adrianN on Thursday December 10, 2020 @01:39PM (#25376832)
We used to pack entire operating systems with userspace in a couple dozen Megabytes. I bet they had more screens than the Uber app.
Re:A former Uber engineer's disaster story by lock-free on Thursday December 10, 2020 @02:59PM (#25377817)
You should scratch below the surface because this isn't really explaining much - it does not sound like something that requires more than 100 people. Their legal team sounds like it should be twice that alone.

Or am I crazy?

Re:A former Uber engineer's disaster story by dntrkv on Thursday December 10, 2020 @03:06PM (#25377893)
Take look at this thread: https://news.ycombinator.com/item?id=25376346

My point is that there is 100s of screens/features in the Uber app that you don't even know exist. And you have those same 100s of screens across their other apps. Multiplied across all of the different platforms. Plus any internal tools they use.

I'm curious, have you worked at a software company of Uber's scale? Not trying to be a dick here, genuinely curious because I hear these types of comments often here on HN and I wonder how many of those commenting have seen the scale from within.

Re:A former Uber engineer's disaster story by lock-free on Thursday December 10, 2020 @04:49PM (#25379200)
My whole point here is I don't understand why "Uber scale" is so large. I have worked in large organizations - smaller than Uber, but we did more stuff (as in more complexity and diversity in products - shipping software and the hardware to run it, for example).

"100s of screens" is as meaningless to me as "thousands of lines of code" and wholly uninteresting because it seems like the front end is the least significant reason why Uber requires so many people to operate. That's why I'm interested in seeing how many people are assigned to projects and what they actually spend their time doing, because at scale you begin to experience inefficiencies due to scale for bad reasons if your incentives aren't aligned. From the original twitter thread that sounds like it was rampant at Uber.

Not sure how long you've been around, but "Uber scale" used to be a joke a few years ago because no one could understand why they would reinvent so many wheels because "they don't work at Uber scale." The joke being that Uber didn't need to be Uber scale to begin with, they just had investor money to burn.

Re:A former Uber engineer's disaster story by sigotirandolas on Friday December 11, 2020 @04:36PM (#25391364)
I'm just a layman, but I often ponder this same "Why are some companies so big when a small team could do about the same thing?" and my mental theory currently is:

1. The amount of engineers scales very badly with respect to application complexity. I.e. you may need 100 or 1000 engineers to do something that "looks" about 2x as complex as something 10 engineers can do, and

2. The more complex solution tends to win in the market because on the one hand people value the application handling edge cases better, having shiny graphics, having slightly better UX, etc. and on the other hand you redeem the cost of paying those 1000 engineers thanks to software being free to copy and network effects / monopoly.

Re:A former Uber engineer's disaster story by ForHackernews on Thursday December 10, 2020 @03:39PM (#25378307)
But like, why? Do all of these things make them tons of money? It's a dumb taxi app, ultimately.

You're not answering the question except as a tautology:

- Q: Why is Uber's engineering team so ridiculous and bloated?

- A: Because they need a ridiculous bloated team to make this ridiculous bloated app.

Re:A former Uber engineer's disaster story by dntrkv on Thursday December 10, 2020 @04:18PM (#25378850)
Just like Alphabet is just a search engine, and Microsoft is just an operating system, and Amazon is just an online store.

Companies grow into new products, services, markets, etc.

Re:A former Uber engineer's disaster story by ForHackernews on Friday December 11, 2020 @07:38AM (#25385049)
Google really is just an advertising company. Everything else "Alphabet" does is a rounding error next to ad revenue.

Microsoft and Amazon have multiple profitable lines of business.

Is Uber more like Google or more like Amazon? Is "Uber Eats" (underpaid contractors drive around takeaway dinners) distinct from "Uber" (underpaid contractors drive around people), or "Uber Black" (underpaid contractors with unwise car leases drive people around)?

Re:A former Uber engineer's disaster story by abenga on Monday December 14, 2020 @07:03AM (#25416287)
> Google really is just an advertising company. Everything else > "Alphabet" does is a rounding error next to ad revenue.

It's not like people would go and visit a page with just a list of ads; no matter how little revenue the other products around ads bring in, they bring eyeballs to the ads, and serve as a moat around ads.

Re:A former Uber engineer's disaster story by abenga on Monday December 14, 2020 @07:04AM (#25416306)
> Google really is just an advertising company. Everything else > "Alphabet" does is a rounding error next to ad revenue.

It's not like people would go and visit a page with just a bunch of ads; no matter how little revenue the other Alphabet products bring in, they bring eyeballs to the ads, and serve as a moat around ads.

Re:A former Uber engineer's disaster story by marcinzm on Thursday December 10, 2020 @12:57PM (#25376378)
At a growing massive company it's not about efficiency but total outcome. If 10x the engineers gives you 10% more revenue then that's worth it economically if the pie is large enough. 10% of 4 billion a year can pay for a lot of engineers. Once you're no longer growing then you need to focus on efficiency as more engineers rarely translates to more total revenue in that case.
Re:A former Uber engineer's disaster story by Justsignedup on Thursday December 10, 2020 @12:53PM (#25376343)
I had similar questions.

Having a large number of engineers is fantastic for making the company valued higher.

Directors love having more engineers under them.

Overall I've been highly speedy with 4 engineers, and slowly adding a 5th. Sure there are limits, but I think limits makes us think about being more efficient with how we build. It is an entirely different mindset.

Once you decide to "go big" you end up in this situation where you usually literally throw a dozen engineers at the problem and the problem goes away. I've seen designers spend 6 months on a _single page_. Because they can. and they get paid crazy money for it.

Re:A former Uber engineer's disaster story by lhorie on Thursday December 10, 2020 @05:43PM (#25379799)
A helpful way to think of it is by comparing how Uber and an OS GUI are different (since that's the comparison people are making, anyways)

With an operating system UI, the developer may reasonably know what the UI will look like, what features are present, where they are placed. With Uber, a single person may not know all the rules for a single feature, and the rules change frequently, sometimes in the most obtuse ways. Like when San Francisco decided to ban cars from one of its most important artery streets earlier this year (market st). Now not only you need to indicate to a rider that this is a thing, your UI also needs to to provide walking instructions to the nearest designated pickup spot, and the routing algorithm needs to not put drivers on market st. Oh, and don't forget Hebrew speaking tourists read right-to-left and that food delivery bicycles _are_ still allowed on market st. That's just one change in one city.

Consider that just about anything might change anywhere, any time. From how earning breakdowns are displayed for drivers in Sao Paulo (this differs by jurisdiction), to where the pickup locations are in Pearson airport in Toronto (airport rules change all the time), to what promos apply in Mexico City this week (which depends on marketing and finance analysis), to whether surges should be disabled in response to a terrorist attack, to what documents a driver signup rep should be scanning into the system as local regulations change.

Now build all the infrastructure required considering that the people on the ground reporting their local changes are not programmers, and conversely many of these changes may require making changes to the UIs that these people use, how the new data is aggregated, etc. There are literally thousands of greenlight hub / customer support employees worldwide using these internal tools, many don't speak english and timezones are all over the place. The nature of problems are often fuzzy, hyperlocal-specific (e.g. certain forms of fraud), or extremely meticulous (e.g. legalese, etc).

The trouble of scale comes with having to deal with numerous nebulously defined things from numerous stakeholders and having to distill which engineering teams are even responsible for which aspects of which thing. As the number of stakeholders grow, so does the risk of miscommunication, unclear chain of responsibility and other well-documented large organizations challenges.

Re:A former Uber engineer's disaster story by lock-free on Thursday December 10, 2020 @11:27PM (#25382284)
That's a bad comparison and I'm not sure why people are making it - you're mistaking fundamentals. This isn't a UI problem, it's datastructures and algorithms. There are plenty of desktop apps with similar issues (representing a complex problem as a set of constraints and writing a solver to find a solution, and querying the solution for the data to display to a user).

I don't reject that there are troubles in concerting real world data into a representation for the constraint solver. But this isn't untrodden ground, and there's no reason a front end developer should need to care about it. The problems you're describing are far behind the UI; and if they're not your organization has immense problems.

Re:A former Uber engineer's disaster story by lhorie on Friday December 11, 2020 @01:13AM (#25382878)
> there's no reason a front end developer should need to care about it

Well, yes and no. You're right that trying to express every nook and cranny of the business rules as solely a UI problem is a wrong way of trying to understand the complexity. But a frontend developer does need to care about how to surface whatever complexity needs to be surfaced, and likewise, backend developers need to know how to model the complexity, and there needs to be some level of orchestration between all the moving parts. Remember, Uber is built on a microservice architecture, and many things that would be arbitrary hardcoded values in projects elsewhere (e.g. pricing) are instead tied to complex dynamic systems in Uber.

In the pricing example, when Uber added the option for drivers to set their own multipliers, it wasn't just a new slider in the driver app. It also affected the pricing AI model, the matching algorithm, how and when prices are displayed in the rider app, geofencing rules, etc etc.

When new regulations mandate that every commit that makes production changes in your city must have an appropriate audit trail, when you need to deploy a cheat-proof face mask detection AI model in response to a global pandemic, when you need to figure out how to make customer support cost $1 per engagement instead of $10, when you need to figure out how to make your network protocol faster in Bangalore, when you need to figure out how to make GPS more accurate in Manhattan, or when any of hundreds of fuzzy problems need solving, then saying that variations of all of these problems have been seen elsewhere misses the point that going from problem statement to deployed solution isn't always a trivial task.

The original question was why so many people work at Uber. The answer is that there are a lot more fuzzy problems than meets the eye.

Re:A former Uber engineer's disaster story by lightroast123 on Thursday December 10, 2020 @02:22PM (#25377367)
We've previously spent a lot of resources also looking for ways to render native components through backend APIs (there was a team called screenflow that was laid off) but this was complex and wasn't able to support complex business needs. Uber does not want to remove app size at the cost of UX (ex. webviews).
Re:A former Uber engineer's disaster story by jonny_eh on Thursday December 10, 2020 @12:44PM (#25376251)
Seems like a reasonable architectural choice early on when your app is live in 2 cities. Instead of changing the language of the app, they should have focused on its API architecture.