Re:Please – A cross-language build system
by liuliu on Saturday November 28, 2020 @05:31PM
(#25241100)
Incremental builds with the said tools (Please, Bazel) should be a non-issue due to emphasis on hermetic build from these tools.
Re:Please – A cross-language build system
by rhencke on Saturday November 28, 2020 @07:58PM
(#25242001)
Unfortunately, reality is messier than this. For example, Please currently has escape hatches that can be used that nullify these guarantees. What I have found is that if such a mechanism exists it will eventually be abused.
My perspective here is specifically from that of a maintainer of large builds where the code base is always under active development. Murphy's law becomes your enemy at scale.
This is not a criticism of Please - just an observation that as an industry we are not there yet. But, I'm happy to see things moving in the right direction.
I haven't had many issues with non-determinism when using Bazel/Blaze? Especially when you use remote execution or sandboxing it's pretty hard to avoid being hermetic.
Re:Please – A cross-language build system
by rhencke on Monday November 30, 2020 @04:39PM
(#25259358)
Here's what I mean:
genrule(name = "oops", cmd = "date > $@", outs = ["oops.txt"])
This is admittedly contrived, but sadly not unrealistic.Again, I am not picking on Bazel here. Correct builds are difficult enough to get right even when everyone involved wants to keep the build correct! But, the reality is that a lot of real-world build conditions can be downright hostile. (did you know that ext4 vs xfs can change the order JARs in the same directory are loaded off the classpath? I wish I didn't!)
Re:Please – A cross-language build system
by tomlu on Tuesday December 08, 2020 @08:44PM
(#25354355)
Yeah fair point. At Google a lot of these non-deterministic rules are either banned through "date" just not existing on remote execution machines, or through determinism tests that ensure rules behave. Maybe in real-life scenarios it's not as easy as I claim, as you suggest.
So much wasted time goes into diagnosis of incorrect results from incremental builds that most people who implement CI systems never use incremental builds, and always build from scratch. Developers are far too used to having to do things like 'make clean' because their build didn't work quite right. Efficiencies are then gained by doing things like ccache/Gradle build cache, which trace their dependencies better than most naively written build systems do.
Edit: zig is a beautiful exception (https://ziglang.org/download/0.3.0/release-notes.html#cachin...)