Hey, thanks for this, I'm going to have to write it down somewhere.
Hopefully my calling attention to it doesn't result in a dogpile of "That's not what 'engineer' means", because the reason I call it out is that it's what I'm usually trying to say when I use the word 'engineer'.
And yet, those are also enormously important problems to solve.
What made Amazon worth billions of dollars to millions of people?
Logistics.
What made Facebook worth billions of dollars to millions of people?
Logistics.
What made Airbnb worth billions of dollars to millions of people?
Logistics.
What made Netflix worth billions of dollars to millions of people?
Logistics.
What made Uber worth billions of dollars to millions of people?
Logistics.
Why did some people think computers were a fad, or the Internet was a fad?
Because they vastly underestimated the tremendous importance of automation, communication, and coordination to mankind.
(An interesting note is that this carries all the dangers of the oft-warned-against 'big rewrite', but on an even larger scale.)
> Facebook ..and being more liberal & exploitative in sharing user data than any company before
> Airbnb ..and subverting local regulations.
> Uber ..and subverting local regulations.
When I was at Google and after I left people would ask me, "How can I get a job at Google?" or "Can you get my resume to someone at Google?" And I would always ask this person "Why do you want to work at Google?"
I can only recall one person, of the dozen or so who asked, that answered the question with something other than (paraphrased) "Because its Google dude, you know the best of the best."
Here is the thing, if the only reason you can come up with for why you should work at Google is because you want to enjoy the free food, schwag, or bus rides then you will have, like the author, a very poor experience.
Then this, "I found such an escape ... through startups.", followed by this "... it soon became clear that the only profitable avenue was to become an advertising technology startup."
Computer scientists in this role are engineers, not researchers. Engineers are people that take existing practice and solve problems that other people are having in a way that is better than the previous solution. If you decide you want to be a "data science" startup then you need to know up front who is going to pay for your data. And the people who will pay? Are most likely ones that can evaluate their own performance with and without your data and get an idea of the difference in income or efficiency etc. This is easy to do with marketing data, you sell more stuff, bueno. You sell less stuff, no bueno. It is more difficult to do with things like recycle awareness campaigns.
If you want to work at a job that has meaning, then work at a company that is solving a real problem. The definition of "real problem" is one that enough people are willing to pay enough money to support a company that is solving that problem. Public Benefit Companies (PBCs) and Non-Government Organizations (NGOs) are solving problems that some organization is willing to fund them to solve it.
That said, during Bubble 2.0 the latecomers especially have lacked a lot of imagination and so are just piling on solving the same problem that others are solving, whether that is intermediating between buyers and sellers (I think of these as virtual market makers) or greasing ecommerce (branding, virality, advertising, shipping/billing logistics, Etc.)