Do teams just work on random projects like the above while other people fix bugs? Is it like a million mini startups internally all pitching an idea to the CEO all the time?
Or do you need 90% of people focused on how to develop your product at first, then suddenly it flips and you need 90% of your engineers to balance enormous amounts of network traffic and data storage?
Many of the projects are just a response to some other internal problem, like "the teams aren't using a shared platform", so they build a platform for the platforms. Persistent problems get ignored because nobody wants to rock the boat. The joys of working in a large corporation...
This is a great way to have engineers burn time on projects they think are interesting rather than projects that will actually have good ROI.
Larger companies with substantial profits can hire engineers explicitly to come up with POCs one after another.
I've seen a lot of vanity projects over the years, and often it feels like this, or a form of in-house retirement. That guy has done so much for us that now we just let him do whatever as long as HR is okay with it.
I do agree there were definitely projects that were overly ambitious and optimistic that likely that should have been better planned. I think you are weighted firmly at one end of the spectrum however. I have some obvious bias being involved in one of the projects you outlined, but I still think you are blindly covering a swathe of projects in one go without any nuance at all.
The project page states "M3, a metrics platform, and M3DB, a distributed time series database, were developed at Uber out of necessity. After using what was available as open source and finding we were unable to use them at our scale due to issues with their reliability, cost and operationally intensive nature we built our own metrics platform piece by piece."[1]
I'm especially curious about Schemaless, since it seemed like at one point every company tried to build a NoSQL KV store on top of sharded MySQL, realized how janky it is, and are currently decommissioning it.
The development team is pretty active. They've been doing an important effort to gain community adoption, e.g. persistency with MySQL, Postgre in the works, Prometheus style metrics, a proposal process or design docs. Very interesting.
I fail to see how.
Considering Uber is reinventing the wheel a lot, the people who were laid off where probably working on some interesting problems. Uber is a fairly well known brand. Unless they are incredibly inapt at selling themselves, their career should do fine.
You're only in a bad bargaining position if you need a job to pay your bills.
The key in the discussion with the recruiter is not to say that you are unemployed but to say that you're taking time for yourself and looking for opportunities that meet your professional desires and baseline financial expectations. The key is to make it clear that you have no financial needs, just financial expectations.
Be prepared to use the Noel Smith-Wenkle salary negotiation method.
I took six months off after a large liquidity event and then went back into the job market. I don't think I lost any negotiating power - the trick was to make sure I times the interviews so that I'd have several offers at the same time and then could use the fact that company X was offering a certain amount to ensure I got the job at the company I liked the most.
I do just want to say one thing: most tech investors now think we are in a bubble. A downturn is likely at some point. Now would be a very good time to ensure you have six months of expenses in a savings account so you don't have to take the first job you get offered - I had to do that in 2008 after the financial crisis hit and my startup died and it wasn't a fun time.
There is a similar effect people have when they reach a certain level of financial independence - not only are offers better, but negotiation is much easier. Companies react to confidence (whatever the source) the same way individuals do.
Also it sounds like Uber needs to fire it's CTO.
Seems like you probably got a lot of shares and were lucky. Now that they IPO'd, do you consider that you came out ahead financially (compared to working elsewhere)?
That said with dilution, it's likely 2015 share price is less than it is today.
I think they moved to RSU's in mid 2015 or 16.
That said, Schemaless isn't going anywhere. Cassandra doesn't cover all the use-cases.
Except the people who were laid off, if they have been there a while, got much deep knowledge than if they just glued together a bunch of AWS stuff.
Gluing together AWS stuff can get you a good job, but it's not the same as solving these sort of problems.
I'd rather spend 3 years working on this and get laid off than the way I spent my last 3. Guarantee I'd be in a much better spot in the job market too.
Instead I'm trying to figure out how the fuck I get my career back on track because I'm headed towards the sort of job I will hate every single day.
But that's just it: the problems were not solved. Personally I would rather devote years of my career building something that actually works by standing on the shoulders of giants, than going full NIH and wasting those years on a clunky POS that will be mothballed before the decade is out.
But hey, I chose science.
I remember doing that too, once upon a time. I invented a templating system based on Lisp for building web apps, back in the days when there were servlets, but no Java Server Pages (or Handlebars, or jsx, or anything we have today).
The templating system was wonderful, and it did improve developer productivity for our group, but there is NO WAY WHATSOEVER that it paid back the time I put into it.
The business would have been much better off if we'd simply used what was available and spent the majority of our time solving business problems, not developer productivity problems.
Furthermore, my templating system didn't wind up being that special. You know why? Because I wasn't really solving a business problem with it, so I had no constraints upon my design. Without constraints, such things meander from shiny bauble to shiny bauble according to the engineer's whims and fancies.
When you're solving an actual problem, you have constraints in time, resources, and solution. This forces you to create actual value. That makes what you build actually useful. But when you're building a thing because you want to play with the thing, you have no constraints, and rather than freeing you to create something amazing, this simply sucks you into making something that is worthless for anything except résumé-building.
"Ah, but my résumé," says the narcissist who feels no obligation to create value for their employer. I have bad news, and I am speaking from experience. If you build a bunch of things with way-cool tech that created no value, you become an ignorant person's idea of a smart developer.
Sure, you can get hired by people who don't really understand how to manage engineering, and they will throw money at you, but their ignorance combined with your self-serving ways will result in another disaster. Lather, rinse, repeat.
Good engineering cultures will ask you about your projects, and they will ferret out the fact that your projects did not add value, and did not "move the needle." I am not hypothesizing this, I have gone to interviews and had the interviewers grill me on my vanity efforts.
You cannot pull the wool over the eyes of anyone who has been through a cycle like this and is determined to never have it happen again. You can put lipstick on a pig, but there's no way to hide the fact that all projects like this wind up becoming ad-hoc, informally-specified, bug-ridden, slow implementations of half of the thing you should have used instead of building your own.
Now all is not doom and gloom. For everyone who has done this, either deliberately or by being dragged into one of these white elephant projects, there is a way to supercharge your career going forward.
Instead of boasting about your ability to write Lisp-based templating systems, or synchronization protocols based on Operational Transformations, or whatever it is you built, try confessing to it.
Point out the engineering experience, but then be up front that the project was a failure/bad idea. Sure, point out that you learned some great tech. But spend much more time talking about how much you learned about failure modes for projects. Talk about the scars.
Companied worth working for are eager to hire people who know better. Position yourself as someone who has learned these lessons the hard way.
Learning those lessons the hard way still requires doing those projects, which means you learn how the tech works, which means I still feel 100% justified in my previous statements.
I want to work on a project like this that actually has lasting value to the company. Actually does something novel enough that it's worth doing.
How do I get there? Is the only way to win the lottery and get placed there as a new grad/junior engineer?
Cause that seems like what you're saying.
I know it's very hard to get jobs in established good companies, but there are also quality startups, founded by people who have these kinds of scars already and won't go down the path of building things nobody needs.
Here's a specific tip:
When interviewing, there's a point where you get to ask them questions. Great! Ask about projects like these. Is the company working on any of its own tech that could otherwise be rented from Amazon or is available as open-source? Why? What is the expected benefit? If not, how do they manage development to stay focused on projects that deliver value?
By asking these questions you will identify engineering groups that are strong or weak, and you will also send them a strong signal that you care about success. Good companies will pick up on that signal and be more inclined to hire you.
JM2C. My advice plus $3.00 will get you a latte at Starbucks.
----
* This is not a dichotomy, of course.
Most of the things we should use instead of building our own are either currently someone's shiny, golden side project with no business case, or else started that way.
Take chat, hipcat is like 5-10 bucks per month per user. Uber has about 20k FTEs, 40k contractors all use chat. That's 3 to 600k per month just on chat.
Don't forget scale is expensive.
I can't imagine how much that many slack licenses would cost.
Can confirm this. It was a very poorly managed org because it was extremely grassroots driven—leadership was underempowered to say no to front line teams or hold groups accountable.
The overlaps and poorly considered projects described above are not an exaggeration. Many of them were designed for building the lead engineer's open source brand, not for company needs. Half of the projects described above have since been canceled.
Google:
- Hyperbahn
- Cherami
- Schemaless
- Peloton
- uDeploy
- m3db
- piper
- XYS
Contrary to their glossy open source and tech blog presentations, these projects were highly contentious internally and widely viewed as inferior to industry counterparts. Each of these projects had 5-20 engineers working on them for over a year; many of them are being EOLed or phased out currently. And this is just the list that I can publicly talk about. This development came at extraordinary cost for the org and, in the case of the people who were laid off, cost to people's careers.