News for hackers. Stuff that matters.
Re:Buck – A build system developed and used by Facebook by beefsack on Tuesday March 14, 2017 @03:12AM (#13865642)
Continuing the trend of Facebook creating competing tools more often than persisting with and improving existing ones, which I feel dilutes effort and has fragmented a number of ecosystems.

We've got a couple of Facebook fans at work, which has left us with a number of our systems using different tools to accomplish essentially the same tasks, and no strong case on either side for us to standardise on one of them.

Maybe I'm just being bitter because I've had a bad experience, but it's been a maintenance nightmare for us in the office.

Re:Buck – A build system developed and used by Facebook by LegNeato on Tuesday March 14, 2017 @04:05AM (#13865839)
When Buck was created (my team created it!) there wasn't anything that supported what Facebook needed in a build tool. There still really isn't as all the Blaze work-a-likes are focusing on various different needs.

I would argue Buck actually helps to solve what you are concerned about. At Facebook, everything builds with Buck (and Google with Bazel I hear)...which means you learn it once and you know it for your Objective-C library, your Android app, your Cxx service, your python scripts, etc. It really helps to standardize on a build system, and at many companies you can only do that if it supports windows/mac/linux, it supports many languages and platforms, is fast, and is easy to pick up. Buck is all of those things.

Re:Buck – A build system developed and used by Facebook by fs111 on Tuesday March 14, 2017 @06:47AM (#13866423)
> At Facebook, everything builds with Buck

Looking at https://github.com/facebook you seem to be using all sorts of build tools...

Re:Buck – A build system developed and used by Facebook by sdwilsh on Tuesday March 14, 2017 @11:56AM (#13868591)
What Facebook uses internally in their monorepo and what they publish to open source for external consumption doesn't have to be the same build tool.

Facebook uses fbshipit (https://github.com/facebook/fbshipit/) to take slices of their monorepo and publish them to GitHub.

Re:Buck – A build system developed and used by Facebook by pjmlp on Tuesday March 14, 2017 @04:56AM (#13866020)
> At Facebook, everything builds with Buck (and Google with Bazel I hear).

Surely not on Android or Skia teams, unless they internally use something other than what AOSP and Skia use.

Re:Buck – A build system developed and used by Facebook by moondev on Tuesday March 14, 2017 @04:29AM (#13865914)
Gradle?
Re:Buck – A build system developed and used by Facebook by sdwilsh on Tuesday March 14, 2017 @11:57AM (#13868604)
Gradle was not the default Android build tool when Buck was open sourced.
Re:Buck – A build system developed and used by Facebook by NeverTrump on Tuesday March 14, 2017 @04:28AM (#13865912)
Any chance you can compare Pants with Buck?
Re:Buck – A build system developed and used by Facebook by skrebbel on Tuesday March 14, 2017 @03:23AM (#13865690)
That's not Facebook's fault, that's fanboyism's fault.

I fail to see how the world gets worse from large companies releasing internal tools as open source.

Re:Buck – A build system developed and used by Facebook by quickben on Tuesday March 14, 2017 @09:03AM (#13867136)
It is very simple. Instead of improving <insert make edition>, they started yet another brand new one.

From my years and years at large companies, these things start because they don't want to share anything in the first place, and then it's just pricey to maintain.

Re:Buck – A build system developed and used by Facebook by BinaryIdiot on Tuesday March 14, 2017 @03:23AM (#13865691)
To be fair a majority of the big technology companies all write their own tooling for many things that already exist because they didn't quite fit the way they needed them to and they had the resources to re-invent whatever they want.

Sure when they open source them it further fragments the market and you always get the rush of "Facebook has almost 2 billion users therefore their tools must be the best!" which further exacerbates the issue but I don't blame Facebook for these issues.

It sounds like, in your situation, perhaps there are too many tools being introduced into your projects. Too often I see developers abuse the hell out of npm and the like to just include whatever they need with zero regard for the newly introduced dependency tree and the new tooling that needs to now be maintained.

Re:Buck – A build system developed and used by Facebook by cwyers on Tuesday March 14, 2017 @03:26AM (#13865703)
When Facebook wrote this, the tooling that already existed wasn't available to them -- ex-Google engineers at Facebook wanted something like Bazel/Blaze, but Google had yet to release that publicly. So this is a Hadoop situation where Google had this internal tool that someone else really wanted and because Google didn't open-source it they wrote their own version.
Re:Buck – A build system developed and used by Facebook by asymmetric on Tuesday March 14, 2017 @04:50AM (#13866002)
Isn't there a risk that reimplementing a previous employer's tool constitutes a legal/contractual violation of some kind?
Re:Buck – A build system developed and used by Facebook by Asooka on Tuesday March 14, 2017 @08:09AM (#13866797)
It's almost certain that it is a legal violation and if a lawsuit was brought, I'd bet a million dollars in favour of Google; however, the tool itself isn't the final product and it's silly, counterproductive and just outright malicious to sabotage efforts at building good tooling in a competitor's company. Plus, it will set precedent and Google would have to vet every single new internal tool against what the previous employers of their employees were doing.
Re:Buck – A build system developed and used by Facebook by whywhywhywhy on Tuesday March 14, 2017 @08:34AM (#13866950)
> To be fair a majority of the big technology companies all write their own tooling for many things that already exist because they didn't quite fit the way they needed them

Too many engineers syndrome comes to mind too.

Re:Buck – A build system developed and used by Facebook by paulddraper on Tuesday March 14, 2017 @03:56AM (#13865803)
What tool would you have suggested Facebook improve?

Saying that Facebook should have improved Make or Maven is like saying Linus should have improved CVS or SVN instead of competing with Git. They're in the same space, but they differ at a very fundamental level.

Re:Buck – A build system developed and used by Facebook by hultner on Tuesday March 14, 2017 @05:45AM (#13866191)
Some of the tools seems more like improving existing tools such as yarn (instead of npm) and jest over jasmine. However I still applaud Facebook for releasing these tools since they often offer massive improvements over previous tools many regards. It might be more apparent if you're exposed to web development since the toolsets are changing constantly and quickly, some stability would be great in this space.

This doesn't mean that we should disencourage creating new tools fundamentally changing the ways we work with them. You mention Git and I see it as on great such example, I remember with anguish some of the SVN trunks, working with their pseudo tags and the mess which you'd often be introduced to.

Re:Buck – A build system developed and used by Facebook by jhasse on Tuesday March 14, 2017 @04:39AM (#13865955)
Waf or Meson
Re:Buck – A build system developed and used by Facebook by paulddraper on Tuesday March 14, 2017 @05:37AM (#13866163)
Facebook doesn't have timetravel, so contributing to Meson would have been impossible.
Re:Buck – A build system developed and used by Facebook by jhasse on Tuesday March 14, 2017 @05:48AM (#13866202)
Ah I didn't know that Buck was created before 2013.

What about Waf though?

Re:Buck – A build system developed and used by Facebook by paulddraper on Tuesday March 14, 2017 @08:37AM (#13866973)
TBH, I never heard of Waf before. It looks interesting.

Buck, Waf, Meson, Bazel....apparently Python is the language for next-gen hash-based build tools :)

Re:Buck – A build system developed and used by Facebook by efficax on Tuesday March 14, 2017 @11:00AM (#13867976)
Buck is primarily Java, just sayin'
Re:Buck – A build system developed and used by Facebook by paulddraper on Tuesday March 14, 2017 @01:11PM (#13869399)
But the syntax of your build files is Pythonish.

Just like Make is in C and the build files are in Make.

Re:Buck – A build system developed and used by Facebook by jacques_chester on Tuesday March 14, 2017 @07:15AM (#13866518)
Facebook and Google have a problem I am familiar with: Not Invented Yet Syndrome.

Typically they face a problem that few people have ever faced. There may be an existing solution, but not in the open. So they have to roll their own.

Others in this thread say Buck was inspired by Blaze. That seems reasonable and I hardly think Facebook can be blamed for rolling their own when that was the only available option.

You might as well blame Google for fragmenting the Hadoop ecosystem by creating their MapReduce framework.

Re:Buck – A build system developed and used by Facebook by perlgeek on Tuesday March 14, 2017 @03:26AM (#13865701)
It seems you lack technical leadership in your organization. Don't blame facebook for it :-)
Re:Buck – A build system developed and used by Facebook by scrame on Tuesday March 14, 2017 @05:05AM (#13866053)
Nope. I totally agree. I don't trust a single thing from facebook. I mean, good on them that they open source their tools, but most everything that comes out is just a huge pain in the ass with very little benefit to adopt, other than we can say we're using open-source facebook technology.

The funny thing, though, is that keeping up with other web tech, you always have to deal with facebooks API's and sharing and whatnot, and its also, usually a huge pain in the ass.

So, like, what's the benefit of this, switching build systems, other than a tick on the resume for someone who wants to get a job at facebook?

Re:Buck – A build system developed and used by Facebook by jlg23 on Tuesday March 14, 2017 @05:12AM (#13866079)
I'm all for using existing tools, but not for the sake of itself. If there is a good reason (and everyone in the team agrees), all now obsolete tools should simply be replaced by the new one.

If this did not work in your office, this is most likely a problem of people in your office not communication properly about their choice of tools. Did you have people make a short presentation when they introduced a new tool? Did nobody ever point out that you already have various tools that can (easily/elegantly) accomplish what the next new thing(tm) is introduced for?

Re:Buck – A build system developed and used by Facebook by thomaslee on Tuesday March 14, 2017 @04:01AM (#13865826)
I can't speak to the rest of Facebook's stuff, but I think the build tool problem is a special case. Per their docs:

> Buck is designed for building multiple deliverables from a single repository (a monorepo) rather than across multiple repositories. It has been Facebook's experience that maintaining dependencies in the same repository makes it easier to ensure that all developers have the correct version of all of the code, and simplifies the process of making atomic commits.

Having been on the "other side" of the monorepo argument where we tried to make do with improving/extending existing build tools etc. in a rapidly growing engineering org, let me say that Facebook (with Buck), Twitter (Pants? I think?) and Google (with Bazel/Blaze) almost certainly built these to deal with the problem of scaling build management with an ever-growing organization.

The popular model of a dozen or so small repositories in GitHub + Jenkins with Maven/NPM/Rake+Bundler/whatever works fine for maybe a few dozen engineers or more, but one day you wake up and realize there hundreds of repositories spread across dozens of _teams_ and hundreds of developers. Obviously you've then got a big ol' dependency graph between repos to deal with, so if you need to fix something near the root suddenly you need to run off bumping version numbers and/or fixing intermediate libraries all the way down the graph. Plus version incompatibilities between the dependencies of different libraries ... it's a total mess, and it doesn't make for an org that can easily "move fast and break things", so to speak.

So then to avoid paralysis your options are basically either to silo up (this team owns their stuff, that team owns their stuff, don't bother with shared dependencies) or you go the monorepo route. If you do, then maybe you go and pull all your hundreds of smaller repos into a monorepo. Having everything in one place makes it easier to police the dependency issues within the org & makes it easier for a single engineer to deal with those sort of "cascading changes" instead of shunting that problem onto the entire organization. But in exchange for this "agility" you've then got the problem that builds take multiple hours & the associated tools are often highly language-centric (Maven+Java, NPM+Node, Ruby+Rake, etc.). They don't typically make any reproducibility guarantees either.

Anyway, to make a short story really long: at the time FB, Google and Twitter were hitting these organizational scaling walls, making these decisions and building these tools internally, there really weren't any great tools out there for the monorepo use case. I think that's why all these tools have appeared as side-by-side alternatives rather than improvements on one another or to tools like Maven et al.

Whether or not consolidation is warranted, for the folks who have the problems that Buck/Bazel/Pants solve, it's likely to save 'em a hell of a lot of time, effort and money IMO. It's a good thing that they have been published, even if the value's maybe not immediately obvious.

Re:Buck – A build system developed and used by Facebook by suls on Tuesday March 14, 2017 @08:43AM (#13867019)
This. Also, I think that the build system itself is just the tip of the iceberg. At least in Google's case it has recently been very nicely documented [1] that blaze is "just" one piece of how google keeps velocity high

[1] https://arxiv.org/abs/1702.01715

Re:Buck – A build system developed and used by Facebook by andrei_belov on Wednesday March 15, 2017 @05:13AM (#13874801)
Great!
Re:Buck – A build system developed and used by Facebook by mhd on Tuesday March 14, 2017 @03:48AM (#13865778)
I'm veering a bit towards the opposite: Some of the Facebook tools look quite nice, but there's a certain amount of taint that goes with their corporate heritage, so if there's a good alternative...