Making a search engine that is good is actually a reasonable task as shown here on HN. It doesn't seem to take very many people, either. Google has set the bar really low.
By contrast, making a search engine that is monetizable and good seems to be an impossible task.
100% agreed.
They really should have used Twitter as an example. Any competent full-stack developer could probably have a working clone of Twitter running in a weekend.
The challenge I think is scaling it to millions of daily active users.
But, even then, Twitter has been around for over 10 years. The scaling problem seems to have been solved. I would think that these days, Twitter engineering would just be a whack-a-mole of odd obscure bugs. I can't think of any useful new features it's gotten in the last two years. Why do they still need so many engineers?
What's a tweet? We can haggle this up and down so cut me some slack. 200 bytes? 50 bytes? gzipped -90%, 5 bytes.
Holy shit what absurd amount of traffic could a 1gb fiber connection handle?
oh huh 200 million tweets a second. That's silly. How wrong could I be? -- then you run down the numbers and you realize that no matter how wrong you are, a twitter-clone-for-most-people oughta be cheap. The network effects and fuzzy human reasons are their moat, nothing else.
One of those fuzzy human reasons is how amazon EC2 eats your ass. Just gobbles it from stem to stern.
The text part is basically solved at this point, but not quite that easy. I hear dealing with blocks is one of the slow parts.
I hazard there's a verdant, halcyon prairie between the two zones. The common zone of maximum extraction, and the hypothetical 200$-a-month-operating-costs text only twitter that can service 5% of the entire planet.
Where I see your engineering headcount blow up is in moderation, legal and advertising; which I think everyone skips over when doing their mental math. Campaign management, targeting, reporting, and optimization on such a large corpus of data is where I struggle to see even a team of 5 building that in a couple of weekends.
OK, but you aren’t just pushing raw ASCII across the wire, you’re getting those results out of a database.
For a weekend clone, someone will probably just use an RDBMS (a very reasonable choice). I’ll be generous and say they can retrieve a given tweet in sub-msec time, say 10 usec. Suddenly, you can only send 100,000 tweets/sec, ignoring all other overhead (which of course you can’t).
I’m sure there is unoptimized code and some unnecessary bloat, but fundamentally there are limits on data retrieval that are very, very hard to get close to.
Think about TikTok: It’s essentially just an app that lets you scroll through videos, yet it became breathtakingly popular just due to the fact that they seem to know exactly which videos you may want to see.
But the point is that this product is not what makes the company money.
If you only consider this, sure it’s a easy to clone product, backoffice for the advertisers, algorithms to maximize engagement, external integrations, marketing, PRs, partnerships are the thing that brings money home.
Of course if there was no Twitter, I imagine you could earn some money/be profitable with just the public product and some basic backend for ads publishers. But you’ll probably encounter a revenue bottleneck and start researching fresh income which you can do by … becoming Twitter, the big company.
I'm an AppSec engineer. I don't write a lot of code, and when I do, it's entirely scripts and occasionally modifying existing code (both front- and back-end, but more often back-end) to implement security fixes.
That said, I could probably hobble it together. It'd be ugly as sin with hand-written Web 1.0 style HTML and likely server-side rendering since I hate writing front-end code, but it really wouldn't be hard as long as you're keeping it simple - Just text posts (No video/pictures, though maybe that wouldn't be too hard to add?), no algorithm for engagement (just chronological ordered tweets), no analytics to determine what's trending. Obviously no mobile app.
The obvious differences being the scale, not really having to worry about the complexities of running your own data centers and overlay networks or whatever, and the fact there is no advertising, no sales, so none of the clutter that comes from trying to optimize that as well as defeat your user's attempts to subvert being tracked, nor any tracking of what you're doing off-site.
The signup process is really nice, though. You don't need individual accounts on each site. They pull your identity automatically from your IC PKI client cert and create the account for you when you first use it.
I work from home now, so I'm not on there any more, but they called the Twitter clone "eChirp" and the Facebook one "Tapioca." They also clone Wikipedia and Stack Overflow, but those are just read-only mirrors that get updated once a day.
And then how are you going to ensure you don't provide content rabbit holes that can be harmful to mental health. Then detect credible cases of people threatening suicide to get a welfare check run.
That's just one of the things you don't see on the surface, there are hundreds more functions like that, that you'll very quickly realise you need to do so you don't get fined out of existence and/or end up with execs going to jail.
That's not to say they are or aren't staffed right, just that the part we are conscious of daily isn't the entire thing.
Does anyone have a link to this?
BKNG: Employees: 23,100, Revenue(2022): 17.1B, Net Income(2022): 3.1B
ABNB: Employees: 6,811, Revenue(2022): 8.4B, Net Income(2022): 1.9B
I can be critical of Google at times but I'll never actually properly come to terms with how Google's search index is still so fast.
One thing to ask yourself - is the company chasing growth? A company like AirBnB or Twitter could absolutely run on a much smaller team if they agreed to mostly abandon growth. They could focus on reducing tech debt, make architectural decisions that reduced flexibility in the name of reliability and lower maintenance, and then eventually lay off a huge portion of their staff.
Instead they have a growth thesis - that growth requires investment, having to favor velocity over unit economics, etc. The higher the growth target, the more inefficiencies you take on.
I think it's hard to argue (although some people clearly did) that making a search engine is easy. You need to index all of Internet, cache it maybe too, rank stuff, filter out spam, work over different languages, do something about images, parse a lot of (broken) HTML, maybe run some js to figure out what a dynamic page should show, then serve it up super quickly... google is faster than pinging a local (!) NFS server at my work.
But then I remember a comparison on HN about how booking.com is O(50) people and Airbnb is O(2000) total. Or I remember the total headcounts at Twitter, an app that lets you post very short messages.
Or LinkedIn, a vast company with a very subpar UX, basically an inferior clone of a dozen of other social networking sites, only kept afloat by its network/monopoly effect. I mean, searching for people by name barely works! You're better off using Google to look for people on LI that its own search box!
No doubt it's just my cognitive limitation, like I can't visualize the vastness of the Cosmos, but this is a close second.