"Crap how do we make money"
"Crap how do we retain users/keep them here longer"
The stack is to do with providing a fast performance to vast numbers + making it easier to work on for engineers than it might be at that complexity.
But there are definitely stack complexity trade offs when you get to the scale of these guys.
Also could be more correlation not causation that when you get to a certain scale and more stack complexity you also start hiring specialized staff that end up diluting the culture and spirit of a startup.
Saying that Facebook is recommending things simultaneously at random, based on a reccomendation algorithm, and also as an optimisation which wouldn't be needed if they used MySQL, is obviously nonsense.
I'm not saying MySQL > some specialized solution for scale... I'm guess I'm drawing the correlation that when companies have to make investments like this they are also trading off experience, or maybe to your point screwing up the experience using recommenders and algorithms instead of letting my feed flow.
- LinkedIn has been terrible for about 2-3 years since things stopped updating real time and the timeline started getting random
- Facebook randomly suggesting things from your graph based on what it thinks you like
- Twitter I heard soon will be no longer in chronological order?!?
I think there is something to be said for low complexity MySQL and memcached.