All they had to do was write a PHP compiler and a new storage engine for MySQL.

HipHop (later HHVM) was around 2010, so they scaled from 2004-2010 before that became needed. MyRocks was 2015. Wikipedia says FB was around 300 million users in 2009, then 400 million users in 2010.

Yes, good point. But you have to wonder what kind of engineering effort went into scaling PHP and MySQL up to the point where they decided to build a compiler and a storage engine.

When you have half a billion users that both read and write all day, you have to optimize, no matter the tech.

That is undeniably true, but I do think the starting point still matters.

It was much, much, much cheaper than a rewrite would have been, that's why they did it.

Edit: also, in 2004 when they got started, what else could they have used?

The trick was there was enough growth that the savings from the compiler were massive. (I worked there at the time.) The inefficiency of the PHP interpreter was a great problem to have, because it came from the success it enabled.

So I think the interesting question is whether the rest of us can learn anything from what happened there.

I believe Mark Zuckerberg simply used the technology he knew and took it from there. That's fine. I probably would have done the same thing.

But many people are making an ideology out of this, arguing that not giving a shit about performance is always the right choice initially because that's how you grow fast enough to be able to retool later.

I think this is based assumptions that are no longer true.

In the early 2000s, mainstream programming languages and runtimes were either fast and low productivity or slow and high productivity (Exceptions such as Pascal/Delphi did exist but they were not mainstream). And the cost of scaling up was prohibitive compared to scaling out.

Today, you can choose any fast high productivity language/runtime and go very far mostly scaling up.

Learn what? That you should use the language that you’re more comfortable with and then scale? Or that languages have become more efficient? Php 8, for example, is many times faster than the php 4 and 5 that Facebook was using.

I think using what you already know remains a choice that is very hard to criticise. But we didn't have to learn that, did we?

Beyond that I think there is more to unlearn than to learn from the history of the Y2K batch of startups. The economics of essentially everything related to writing, running and distributing software have changed completely.

In part the reason that PHP8 (and it was 7 that had the quantum leap in perf) are now so fast is precisely because of Hack - it was easy to accept the status quo on performance until Hack showed there really was a lot of performance left on the table.

For me the biggest win was the changes they made to how arrays are stored in memory, I saw some systems drop by half in memory usage and had to change basically nothing - those kinds of wins are rare.

Yeah, I know the performance optimization were in part because of hhvm.

I take away two lessons from it, which are in a kind of essential tension that can only be mediated by wisdom and experience:

1) Pick technologies that you can optimize.

2) Don't over-optimize.

Also, the concept of "optimization" here has very little to do with the language itself. It's far more about the overall stack, and it definitely includes people and processes (like hiring). It's not like FB invested $0 toward performance before swapping out PHP interpreters! Its massive caching layer, for example, was already taking shape well before HPHP (the C++ transpiler which preceded HipHop), not to mention the effort and tooling behind the MySQL sharding and multi-region that still exists in some form today. Many backend FB services were already written in C++ by 2010. But they had already gone very, very far—farther than most businesses ever will—on "just" PHP. Heroics like HPHP only happened after enormous pipelines of money were already flowing into the company.

Did they succeed because of PHP or was it just a tech used at the time and anything else similar at the time would be fine either way?

They succeeded because of php. It was easy to use for them. So it enabled them to materialize their ideas. It was the right tool for them. Anything else would have been fine either way, if it was the language they were the most comfortable with. In their case, it happened to be php.

Well, OK. But by that logic, if the language they had been most familiar with was Fortran, should they have used Fortran for Facebook? I tend to think that there are actually material differences between languages and technologies, and it's worth knowing more than one language and not using terrible ones.

“ But by that logic, if the language they had been most familiar with was Fortran, should they have used Fortran for Facebook”

Absolutely. Otherwise they wouldn’t have been able to release the actual product and keep adding features to it the way they did with Facebook. They’d spend half the time learning the “right” language & environment. That would have slowed them down to the point they wouldn’t have been able to work on the actual product as much as they did.

And feature-wise, Facebook evolved really quickly.

That just sounds like "they succeeded because they knew a programming language", not that it was right one compared to competition

No, they totally succeeded because the used php. I think zuckerberg said it himself that php allowed them to add new features easily. I think he mentioned that it was easy for new people to pick it up. I’m pretty sure Facebook wouldn’t exist today if it had been written in the more corporate/esoteric languages available at the time.

Its ease of use allowed him to launch the site from his dorm room. Iirc, YouTube was also written in php (it had .php urls), before google bought it and rewrote it using python, so you could probably thank php for that site too.

Just checked. It appears it was indeed first written in php then changed to python then to java.

I don't think there was anything similar to PHP that wasn't proprietary (Cold Fusion etc.), and FB engineering culture was to avoid vendor lock-in.

In any case, in the 2000s a PHP programmer/designer was analogous to a JavaScript developer today. Lots of talent out there, and it only took a few weeks of orientation and familiarizing for new hires to be productive.

your comment implies your understanding of the timeline is backwards. they had to do those things after they had gotten hundreds of millions of users.

Depends on what you consider "far down the road" and what they had to do before writing a compiler and a storage engine.

How long did it take until Facebook engineers realised that their technology stack was not the best tool for the job? It definitely wasn't the day when they decided to build a compiler and a storage engine.

I'm not sure there was really a best tool for the job in 2003-2004 that would have been high-level enough to be productive, and scalable enough to stay mostly as-is. Java, maybe.

I agree, and I'm not criticising the choices that Mark Zuckerberg made at the time. But we are no longer facing the same situation he did. We do now have high productivity, high performance language runtimes. And scaling up has become much cheaper (relative to the number of users you can serve).

That's why I think it can't hurt to remind people of the great lengths to which Facebook had to go in order to deal with the limitations of their chosen platform.

Yeah, I kinda don’t agree with the dichotomy of “you either optimize or you build features”. They’re not exclusive. If you understand the tools and their trade offs you should be able to use the right tools for the job which won’t hinder you in the future.

Of course if all you know is JavaScript, then that requires going a bit outside your comfort zone.