News for hackers. Stuff that matters.
Re:Buck: A high-performance build tool by appleflaxen on Sunday April 21, 2019 @08:07AM (#19711701)
why are there so many build systems?

it's understandable why programmers have opinions about aesthetics regarding their IDE, language, or framework of choice, but what is there to be opinionated about with a build system?

Re:Buck: A high-performance build tool by jasode on Sunday April 21, 2019 @08:42AM (#19711827)
>, but what is there to be opinionated about with a build system?

I think you're saying this because your idea of a "build system" is a very simple set of sequential steps from known source code input files to an output binary.

The differing opinions come in when the "build system" includes various contradictory philosophies of how to configure and specify the building of complex software.

Different opinions on:

- syntax : should build config be XML, or JSON, or YAML, or custom syntax? E.g. Ant and Maven used XML but Gradle does not

- dependencies search: should build system implicitly find and add relevant dependencies? Or should programmer explicitly specify each one?

- reproducible builds (version pinning) vs auto-updated dependencies.

- should the build system be "smart" about cross-platform differences? If yes, you end up with complicated build tools like GNU Autoconf "configure" bash script, or CMake's complex syntax.

- should build system do optimization and bundling tasks that's not strictly limited to "compile" steps? Some Javascript build tools try to eliminate redundant or unused js code to make downloads smaller.

- etc, etc.

I also recommend reading the blog post "So you want to write a package manager"[1] to get an idea of how complicated a build system can get. The title says "package manager" but much of the material is also about build systems.

The bottom line is that reasonable people can disagree on the priorities and therefore, you can't create The One & Only Build System to End All Other Build Systems.

[1] https://medium.com/@sdboyer/so-you-want-to-write-a-package-m...

Re:Buck: A high-performance build tool by Aeolun on Sunday April 21, 2019 @08:21PM (#19715428)
I feel like there is a ‘best’ answer to most if not all of these questions, and the build system should standardize on whatever that best practice is.

It’s great that Timmy thinks an obscure extension of YAML is the best way to define build configs, but all the rest of the engineers use JSON, so that is what we use.

Re:Buck: A high-performance build tool by woadwarrior01 on Sunday April 21, 2019 @08:20AM (#19711751)
Because Google didn't open source Blaze soon enough, and Xooglers who moved to other companies wanted something similar and ended up building clones (Buck and Pants come to mind). Eventually, Google did release an open source version of Blaze called Bazel, but by then the eco-system was already rife with many other similar but slightly different build systems.
Re:Buck: A high-performance build tool by geezerjay on Sunday April 21, 2019 @09:00AM (#19711887)
> why are there so many build systems?

Perhaps because the incumbent/popular build systems are far from being the best solution for the job, particularly when compared with build systems available for other technologies.

For instance, in C and C++ land the incumbents are still hand-written Makefiles, autotools, or cmake, which leave much to be desired.

Re:Buck: A high-performance build tool by detaro on Sunday April 21, 2019 @08:33AM (#19711796)
- configuration format/language

- system requirements

- convention vs configuration

- flexibility vs predefined paths

- tradeoff features vs their complexity (parallelism, caching, ...)

- level of integration with specific other tooling (VCS, languages, package managers, ...)

Re:Buck: A high-performance build tool by pytester on Sunday April 21, 2019 @08:24AM (#19711764)
It's a hard thing to do right despite looking deceptively simple.
Re:Buck: A high-performance build tool by blattimwind on Sunday April 21, 2019 @09:26AM (#19711983)
Well many build systems don't even try to work properly, many new ones included. E.g. anything using timestamps is unequivocally an incorrect build system.
Re:Buck: A high-performance build tool by ek750 on Sunday April 21, 2019 @11:36AM (#19712544)
Can you explain why? I’d be interested to learn more.
Re:Buck: A high-performance build tool by ithkuil on Sunday April 21, 2019 @12:30PM (#19712884)
Apart from issues pertaining timestamp granularity and clock skew, there is a problem that is technically not about timestamps but is related since it's often a consequence of the dependency data model employed by that class of build systems.

Consider this makefile rule:

foo.a: $(patsubst %.c,%.o,$(wildcard *.c))

Now, the foo.a target will be considered as stale if any of timestamps of it's dependencies is newer than foo.a

But what if you remove a .c file?

As build systems like make don't capture a fingerprint of the names of the inputs of a built (let alone the hash of their contents) it's very hard for them to deal with that case correctly.

Correctness is important otherwise your trust in the incremental builds quicky erodes and you start doing clean builds all the times you get an error, just in case.

Re:Buck: A high-performance build tool by qznc on Sunday April 21, 2019 @02:06PM (#19713526)
According to Apenwarr (redo implementer) claims [0] that checksums are not necessary, if you extend the mtime with size, inode number, file mode, owner uid, owner gid, and (targets only) the sequence number of the last time it was built.

The blog post [0] also contains arguments against checksums: Sometimes building a target has side effects. Checksumming every output after building it is somewhat slow. Checksumming every input file before building is very slow.

[0] https://apenwarr.ca/log/20181113

Re:Buck: A high-performance build tool by joshuamorton on Sunday April 21, 2019 @02:28PM (#19713662)
Building a Target shouldn't have side effects. But things to may want to do with your build system might have side effects. Bazel for example supports with with the bazel run command, as opposed to bazel build.
Re:Buck: A high-performance build tool by ithkuil on Sunday April 21, 2019 @02:32PM (#19713681)
Yeah, but while the mtime++ vs content hash dichotomy is interesting and all, in a way it's all just about a performance optimization.

What matters is keeping track of the actual dependencies and Redo does indeed save that knowledge in a database that gets consulted between runs.

Re:Buck: A high-performance build tool by lhorie on Sunday April 21, 2019 @11:43AM (#19712583)
In principle, you want the output of a build to always be the same if the inputs are the same. In terms of correctness, you want "sameness" to be defined in terms of the contents of the input, not their timestamp, since timestamps can be easily be changed inadvertently by things like `touch`, 3rd party tools, etc. Also, relying on timestamps could pose problems for caching/checksums/etc if they are printed into any transitive dependency of a build pipeline.
Re:Buck: A high-performance build tool by ithkuil on Sunday April 21, 2019 @12:03PM (#19712689)
Furthermore, timestamp precision is still problematic. Some filesystems still have one second granularity, but even if you use a filesystems capable of storing sub-millisecond timrstamps, that doesn't mean commands that get invoked as part of the build process (e.g. cp -p) invoke the right APIs that make use of such precision. e.g. see the doc for the .LOW_RESOLUTION_TIME directive of GNU make
Re:Buck: A high-performance build tool by falsedan on Sunday April 21, 2019 @08:28AM (#19711774)
Because it’s easy to implement something that does 80% of the features, and also it’s easier than learning all the gnarly features of make

Last year I learned about .PRECIOUS and it was awful

Re:Buck: A high-performance build tool by amelius on Sunday April 21, 2019 @08:24AM (#19711761)
I think it boils down to taste, just as there are many languages which have basically the same expressive power. Similarly, there are many shells like bash, tcsh, ...
Re:Buck: A high-performance build tool by abathologist on Sunday April 21, 2019 @08:42AM (#19711828)
Building and deploying software is an unsolved problem and this bottleneck becomes increasingly costly and painful as the digital revolution accelerates.
Re:Buck: A high-performance build tool by foobarbazetc on Sunday April 21, 2019 @08:54AM (#19711868)
Because it’s fun to build them and “do it right”... :)
Re:Buck: A high-performance build tool by gbrown_ on Sunday April 21, 2019 @08:26AM (#19711768)
Re:Buck: A high-performance build tool by entelechy on Sunday April 21, 2019 @09:56AM (#19712079)
The C++ community started a comparison of various buildsystem a while ago: https://docs.google.com/document/d/1y5ZD8ETyGtxCmtT9dIMDTnWw...