Pedantically, codesearch is also grep. But codesearch calls out to Kythe (nee Grok) which has a semantic graph of the code.
But.... internally everyone thinks the Kythe team is just codesearch anyways, so that's about right lol.
Kythe is partially open source, but critically a lot of the postprocessing to get it to work at massive monorepo scale is not, so FB would have a bunch of work to do to replicating it.
Also we don't have a PHP indexer, because nobody's written one.
What's nice about this scheme at Google is all manner of generated code is indexed, so you can navigate up and down the caller-callee graph between artisanal code and generated code. Really works well.
Even then, it's not like rewriting a billion lines of BUCK files to use Bazel was even practical. Realistically any solution had to have a direct migration path from Buck1 without rewriting everything. I don't even work there, this is just pretty obvious from the design constraints and talking with the dev team, though. Frankly, I'm pretty impressed they were both able to meet the goals they had (migration from Buck1, better performance, more extensible), while still jam-packing the thing full of good design decisions and features like they have. It's good work.
(I am a pretty happy, non-Facebook user of Buck2 already, FWIW.)
My issue is more organizational. Meta, as a company, in my experience, does not put in sufficient investment to build a mature, robust open source project with few exceptions. Even things like the www test infra, which are core to the company when I was there had ~1 FTE SWE. That's not open source but you get my point. Buck (and Thrift) seem to have been woefully underinvested for years. Thrift was originally an intern project at a time when Stubby (Google's protobuf-based RPC) was not open source.
Buck2 is much faster than Buck and pretty great for the set of problems it's trying to solve. (I'm skeptical that the corner that Meta has painted itself into is good, but assuming that that can't change, buck2 is great.)
Remote builds are common now, from what I've heard.
Sapling is the name for Meta's fork of Mercurial. Piper is not a reimplementation of Perforce, by the way. The closest equivalent that Meta has to Piper is their source control server Mononoke.
Yes, it is. You are probably confusing Piper, the internal name for the VCS, with Google Piper [1], which is a completely separate and unrelated project. That project seems a closer match to Monomake.
This was a common problem with Google, actually. There would be an internal name but once released it would take on a different name. Sometimes that name would conflict with a different internal name, which then made internal searches impossible. I forget the specifics, but Buzz became an external product but was an internal product name for something different.
Dart (the language) was internally known as Dash prior to release. IIRC Google Dash was an external thing for advertisers or something like that.
So when I say Piper I really do mean the internal rewrite of Perforce.
> FB's code search is just grep. It's literally called BigGrep.
This hasn't been true for a long time, codesearch at FB is more complex than just grep and has some semantic understanding. Here's some discussion here when some of this infrastructure was open sourced: https://news.ycombinator.com/item?id=28365880
> FB C++ might take 2 minutes just to load the DAG before it builds anything.
Yeah, this is pretty terrible, but is mostly a description that doesn't apply to buck2.
> This is essentially instant at Google because a lot of infrastructure has been built to solve this problem. This is a combination of SrcFS and ObjFS.
SrcFS and ObjFS aren't what solve this problem at Google. And to the extent that they do, FB's sapling and buck integration do the same.
> All non-www builds at FB are local builds. Nothing at Google (on Google3 at least) is built locally, including mobile apps. This is way faster because of build artifact cachcing and you have beefier build machines.
This is wrong since at least 2015 FB's build system had the build artifact caching, and you can see in buck's git history that they've had remote execution just like Google (in fact, using the same RE api as bazel) since like 5 years ago.
I did spend way more time doing www than C++ though and Hack was specifically written to facilitate regex searches like being able to search for "SomeClassName::someFunctionName" to find usages as well as the other examples of prohibiting namespacing and type aliasing (in Hack). Google cs doesn't have that constraint.
> SrcFS and ObjFS aren't what solve this problem at Google
It's a mix. You can't just pull out one piece of the Google dev infra because it's all connected. In a P4 client, you'd list some paths that were "local" allowing local modifications. Forge, SrcFS, ObjFS, TAP and Sponge are all pieces of this puzzle.
> This is wrong since at least 2015 FB's build system had the build artifact caching
First, there's more to FB builds than infra C++, most notably iOS and Android, which are all built locally. It's why iOS/Android engineers have big, chunky machines like the iMac Pro or the trash can. If this has changed, it's a fairly recent change. Google builds mobile apps on Forge. There are literally racks of Mac Minis to build iOS. With this you can build artifact cachincg like you do with, say, Google3 Java or C++.
Second, "local" requires some further explanation. Typically, things are built on a devserver (although this was transitioning to on-demand VMs, for which the Google equivalent was CitC). But a devserver build required a full checkout and build with artifact caching. It could then be incremental until an hg pull forced a larger rebuild. There were sparse checkouts but I think the support was pretty limited and only worked on certain infra projects.
Either way, the whole FB C++ build experience was fairly primitive and even worse for iOS/Android.
Again, ios and Android have had the remote artifact caching you mention as so important since many years ago. Android has had remote execution for years (I think even before the infra c++ you mention).
> But a devserver build required a full checkout
This is not true with sapling, which had been used extensively for years.
Flatly untrue, on iOS much of the infrastructure was well ahead of what anyone else had aside from Google. It worked so well that a tooling team responsible for upgrading to a new version of Xcode soon after its released declared victory and took credit for the compiler upgrade, as this had been an issue in past years.
They hadn't realized that a compiler team had been quietly running their builds company-wide for nearly two years and fixing compiler bugs on the bleeding edge of clang/llvm the whole time, so by the time the compiler was branched for Xcode and released from Apple, the fixes were already complete, open-sourced, and merged fully upstream.
I skipped over that parenthetical in first read, but my God, it's not even wrong. I mean citc is excellent and all, but describing those as equivalent just makes me wonder if you held some non technical role and are just playing telephone from people who better understand things. Like I'm just imagining a conversation where you're like, how does Facebook handle this specific thing that's handled by a X at Google (or the reverse)? Getting an answer on one side that involves citc and the other that involves on demand and deciding that that specific problem represents the entirety of those two ecosystems.
I guess one of the difficulties I have here is that a lot of what you've described across a bunch of google-facebook comparison is just like so factually wrong that i can't ignore the problems with a direct reading of what you are saying. At a deeper level, I think I'd totally accept google's CITC and facebook's on-demand as like philosophically similar in that solutions to seemingly unrelated things fit into the structure enabled by those systems. Again though, I think crediting you with trying to discuss things in those terms would be too generous given that that equivalence is no more true of on-demand VMs than it is of devservers.
> I'm honestly not sure what sapling is
My rough understanding: arcanist + git evolved into arcanist + hg, hg became a frontend to cHg and then the whole edenfs/sapling stuff started to replace that to optimize sparse checkout workflows.
> - Buck should've been killed and replaced by Bazel. I can't speak to Buck2 but this seems like a pointless investment;
Bazel's open-source version remained significantly less capable than Buck for a long time; Buck migrated towards Skylark just like Blaze, and cleaned things up greatly.
> - Thrift should be killed and replaced with gRPC/Protobuf. Same deal;
In general, agree with the former, but that's because protobufs and especially flatbuffers were already used in many places for many many years.
> FB's code search is just grep
There was also a less popular semantic search that people didn't use nearly as much.
> There's no dependency declaration at all
Strictly speaking this is not true, it was just not usually needed.
> There is no way to say "if this file is modified, run this test".
You needed to update the target determinator, but I do think this was possible.
The tl;dr is that Google's developer tooling and ifnrastructure is superior in almost every way. Examples:
- When I started at FB we used Nuclide, an internal fork of the Atom editor. While I was there it was replaced by VS Code. It's better but honestly they should've built their tooling off of Jetbrains products. Jetbrains make IDEs. VS Code is a text editor like vim or emacs. There's a massive difference;
- Buck should've been killed and replaced by Bazel. I can't speak to Buck2 but this seems like a pointless investment;
- Thrift should be killed and replaced with gRPC/Protobuf. Same deal;
- FB's code search is just grep. It's literally called BigGrep. Grep can get you pretty far but it's just not the same as something with semantic understanding. Google has codesearch, which does understand code, and it's miles ahead. This has all sorts of weird side effects too, like Hack code at FB can't use namespaces or type aliasing because then grep wouldn't be able to find it. When there were name conflicts you'd sometimes be forced to rename something to get something to compile;
- Tupperware (FB's container system) is a pale shadow of Borg;
- Pushing www code at FB is a very good experience overall. You commit something and it'll get pushed to production possibly within an hour or two or, at busier times, it might take to the next day. This requires no release process or manual build. It's basically automatic; Google's build and release process tends to be way more onerous;
- The big achillees heel in FB's www code is that it is one giant binary. There's no dependency declaration at all. This means there's an automatic system to detect if your change affects other things and that process often fails. This leads to trunk getting broken. A lot.
- Because of the above problem there is a system to determine what tests to run for a given commit. This is partially about what the affected components are but also longer-running tests aren't run-on-commit and often those tests would've found the problem. There is no way to say "if this file is modified, run this test". That's a huge problem;
- FB has a consistent system for running experiements and having features behind flags (ie gatekeeper). This wasn't the case when I was at Google. It may well have changed;
- Creating a UI for an internal tool or a new page is incredibly easy at FB. There are standard components with the correct styling for everything. If you want to write an internal tool, you can start at 9am and have it in production by noon if it's not terribly complicated;
- The build system for C++ at FB is, well, trash. For Buck (and Bazel), the build system creates a DAG of the build artifacts to decide what to build. FB C++ might take 2 minutes just to load the DAG before it builds anything. This is essentially instant at Google because a lot of infrastructure has been built to solve this problem. This is a combination of SrcFS and ObjFS. Incremental builds at FB to run tests doesn't really work as a workflow;
- All non-www builds at FB are local builds. Nothing at Google (on Google3 at least) is built locally, including mobile apps. This is way faster because of build artifact cachcing and you have beefier build machines.
- There tends to be less choices as to what to use for FB code (eg storage systems). I consider this largely a good thing. You will typically find 5 different way of doing anything at Google and then need to consider why. You will often find different teams solving the same problem in slightly different ways or even the exact same way.
- There are people at FB who work on system-wide refactors (eg Web security, storage). These people can often only commit their diffs that might touch thousnads of files on weekends.
- A lot of generated code is committed at FB that isn't at Google. This exacerbates the previous problem. FB has a ton of partially and completely generated files that mean a change to the generating code has a massive effect. At Google, for example, the protobuf generated code is genearted at build time and isn't in the repo.
There's probably more but that's what comes to mind.