News for hackers. Stuff that matters.
Re:Velox: Meta's Unified Execution Engine [pdf] by sgt101 on Monday March 25, 2024 @05:39PM (#39821541)
I wonder how many of this sort of FAANG project really get used where they are built. I went for an interview at a FAANG years ago to work on a very big consumer product (when it was in relative infancy) and expected to find a hyper tech data backend to use... they told me that they were using mySQL.

I didn't get the job so maybe they were just joking around with me - but the general despair that they evinced about their data situation makes me wonder!

Re:Velox: Meta's Unified Execution Engine [pdf] by ipsum2 on Monday March 25, 2024 @05:49PM (#39821627)
Facebook/meta uses mySQL, but with a completely different engine (myrocks) and sharding techniques.

YouTube uses mySQL but they've also rewritten major portions for scalability. (Vitess)

Just because a company is using a technology you've heard of doesn't mean it's what you expect.

Re:Velox: Meta's Unified Execution Engine [pdf] by emptysea on Monday March 25, 2024 @07:07PM (#39822279)
YouTube uses Spanner now, they migrated off Vitess a while ago
Re:Velox: Meta's Unified Execution Engine [pdf] by wferrell on Tuesday March 26, 2024 @01:55AM (#39824622)
Can you share why they migrated?

Vitess seemed to be working for them for a long time.

Re:Velox: Meta's Unified Execution Engine [pdf] by harshitgangal on Tuesday March 26, 2024 @02:50AM (#39824869)
When Vitess was donated to CNCF, there was an internal push from within Google to migrate everything to Spanner.

I do not know any other reason.

Disclaimer: Vitess Maintainer

Re:Velox: Meta's Unified Execution Engine [pdf] by riku_iki on Monday March 25, 2024 @05:57PM (#39821706)
> YouTube uses mySQL but they've also rewritten major portions for scalability. (Vitess)

I imagine this is some very old info(like 10 yo) and could change since then?

Re:Velox: Meta's Unified Execution Engine [pdf] by philjohn on Monday March 25, 2024 @06:15PM (#39821844)
At Meta they probably don't get built unless they're impactful, and they're not impactful if they're not used in production to solve a real pain point.
Re:Velox: Meta's Unified Execution Engine [pdf] by manzanarama on Monday March 25, 2024 @08:01PM (#39822710)
More like, everything gets build because someone wants to get promoted.
Re:Velox: Meta's Unified Execution Engine [pdf] by bigcat12345678 on Monday March 25, 2024 @08:15PM (#39822833)
That's just some urban myth about promotion in big corp

Yes

There are a lot of vanity projects that get someone promoted for the wrong reasons

That only get broadcasted because that's the newsworthy. You won't get up voted when you share a small story about someone did hard problem and get promoted.

Overwhelmingly, people get promoted because they solve challenging problems with meaningful impact. That's how capitalism and modern corporation work.

But above the baseline there is a lot of errors, exceptions, and manipulations. Because that's how people do everyday: they want to game the system for their own gains. Human nature. There are just so many of them because big corps are big. And that's why big corps eventually lost their vigor.

The best way to combat promotion bullshit and other corporate bullshit, it's to recognize them, call them out in the right technique (being diplomatic and protect yourself) and don't practice yourself.

Yes, don't practice the bullshit. That's extraordinarily difficult.

Re:Velox: Meta's Unified Execution Engine [pdf] by fshbbdssbbgdd on Monday March 25, 2024 @08:26PM (#39822928)
I think it’s true both that most promotions are legit and not based on vanity projects, and yet still the vanity projects are common and causing major problems. Let’s say you have 10k engineers at your megacorp. Maybe the ideal number of execution platform workflow framework engines your business needs to add this year is 30, but instead 300 are created by 3% of your engineers who wants a promotion. Eventually you have thousands of these frameworks, maintaining them is a drag, everyone is suffering, although the vast majority are good actors.
Re:Velox: Meta's Unified Execution Engine [pdf] by valicord on Monday March 25, 2024 @09:29PM (#39823380)
Speaking from personal experience, the inverse of this is not necessary great either: the desire for the ever-growing scope leads to convincing everyone to switch to the "one true system" where previously multiple custom solutions were better fit for each individual problem.
Re:Velox: Meta's Unified Execution Engine [pdf] by threeseed on Monday March 25, 2024 @09:53PM (#39823507)
These projects don't just appear out of thin air and get funded.

It's because solutions don't exist to meet their unique requirements which often you don't get visibility of unless you're in the team.

But of course that will never stop HN commenters assuming they know better about the situation than the engineers and managers that work there.

Re:Velox: Meta's Unified Execution Engine [pdf] by throw_nbvc1234 on Monday March 25, 2024 @09:48PM (#39823475)
> Overwhelmingly, people get promoted because they solve challenging problems with meaningful impact. That's how capitalism and modern corporation work.

It's key to ask, does the promotion (or strong performance rating) happen before the impact or after?

You can deliver Project X that will save $YY Million dollars. Everyone agrees the impact is "there", the complexity is there. Launch a PoC to a handful of use cases, realize most of that impact, then move onto something else. PoC works for those use cases, never becomes a complete solution, and slowly develops issues. Once it has enough issues, someone else can solve the problem again for the even more impact assuming the problem space has grown since the initial launch.

Capitalism works when there's competition and cost for (long-term) failure. Neither are guaranteed to exist if you're at a Big Corp that's printing money.

Re:Velox: Meta's Unified Execution Engine [pdf] by bigcat12345678 on Monday March 25, 2024 @11:09PM (#39823937)
》Capitalism works when there's competition and cost for (long-term) failure. Neither are guaranteed to exist if you're at a Big Corp that's printing money.

Disagree

Big Corp print money still squeeze employees. See the record profit & revenue and 10k+ layoffs.

Re:Velox: Meta's Unified Execution Engine [pdf] by hodgesrm on Tuesday March 26, 2024 @08:44PM (#39834547)
That's a pretty cynical take. Meta deployed Facebook at enormous scale as in many thousands of MySQL servers. The engineering team included a number of the best engineers in the MySQL community, who adapted MySQL extensively to meet the needs of Facebook applications. They used MySQL because it worked.
Re:Velox: Meta's Unified Execution Engine [pdf] by kgp7 on Monday March 25, 2024 @06:19PM (#39821862)
This is being actively used at Meta in Production across several engines ; the paper makes explicit references to this.
Re:Velox: Meta's Unified Execution Engine [pdf] by bezosdontpipme on Monday March 25, 2024 @05:43PM (#39821574)
I can neither confirm nor deny that S3’s global bucket database is actually just MySql (with a lil bit of special sauce)
Re:Velox: Meta's Unified Execution Engine [pdf] by nonrandomstring on Monday March 25, 2024 @06:01PM (#39821738)
And why ever not? It's a perfectly good solution, no?

What the GP alludes to is interesting though - mythologising of organisations, brands and names.

Spend enough time with "famous" people, "big names", centres of power and prominence and you quickly see everyone is just ordinary dudes doing ordinary things with ordinary gear. But for some reason there's fuck loads of money and attention, and sometimes cloying paranoia and adulation floating around.

Sure, right out on the periphery are a noble few who play with particle accelerators, spaceships and bunker supercomputers. But then, that's just a day job too.

True genius/exceptionalism is rare and found in the unexpected places. The rest is conjured out of thin air by marketing and PR people, the press, and commentators. They are the ones who need the big legend.

Re:Velox: Meta's Unified Execution Engine [pdf] by influx on Monday March 25, 2024 @06:02PM (#39821742)
Yeah, but I bet you the S3 Keymap isn't MySQL....
Re:Velox: Meta's Unified Execution Engine [pdf] by sgt101 on Monday March 25, 2024 @05:47PM (#39821610)
tbh my general response to all data questions is "use postgres". It does happen that someone comes back with a good reason why that would be a bad idea, but it's not frequent!

mySQL == Oracle now... so bad on theological grounds.

Re:Velox: Meta's Unified Execution Engine [pdf] by astrange on Monday March 25, 2024 @06:04PM (#39821753)
You can use MariaDB.