Re:Towards Idempotent Rebuilds?
by orbital-decay on Wednesday July 10, 2024 @03:58PM
(#40930872)
Multithreaded code can easily be non-deterministic, with things like OS preemption leaking into the sandbox. Guaranteed determinism is hard, it needs a deterministic emulator.
Not really for building code, at least. It's a common problem, but it's not insurmountable. Use the equivalent of 'make -j1' or whatever...
This requires engineering on the part of compiler writers, but ultimately is solvable, given enough funding.
(About the compiler writers part: I'm thinking about GHC Haskell and the like... the Clangs/GCCs of the world will have no problems because of Translation Units. Things may change when Modules are a practical choice.)
You're gonna need a better programming language if you want to get to that level.
Re:Towards Idempotent Rebuilds?
by bobmcnamara on Wednesday July 10, 2024 @02:07PM
(#40929665)
For these sorts of things, please do not assume hardware is reliable and compilers are bug free.
Re:Towards Idempotent Rebuilds?
by 8organicbits on Wednesday July 10, 2024 @03:06PM
(#40930269)
Bug free is less important for reproducible builds, so long as the bugs are reproducible. Compiler bugs are of course still a problem for other reasons.
Re:Towards Idempotent Rebuilds?
by bobmcnamara on Wednesday July 10, 2024 @03:39PM
(#40930659)
My point is only that even when the software is correct, the reproducible build rate will either be 100% or we've got a representative sample set.
Until then, those issues will continue to be lumped together with all the software reasons it wasn't reproducible.
Sounds like a mechanism to test for bad hardware (akin to memtest86) and find compiler bugs =)
Re:Towards Idempotent Rebuilds?
by bobmcnamara on Wednesday July 10, 2024 @03:08PM
(#40930295)
Bingo - even if we do everything right the percentage of matching builds isn't expected to be 100%. If it is, there just aren't enough samples.
My first Pentium could usually but not always math correctly until Intel replaced it during a recall.
Re:Towards Idempotent Rebuilds?
by yjftsjthsd-h on Wednesday July 10, 2024 @09:17PM
(#40932978)
In a slightly different form, this has already happened: https://julialang.org/blog/2020/09/rr-memory-magic/
> There is always going to be a degree of un-reproducibility just due to the nature of math.
Fundamentally the math is deterministic (reproducible): if you do the same sequence of mathematical operations you get the same result. I gather you're getting at the non-associativity of floating point (eg additions), which is a fair point, but if you arrange to do your floating point operations in the same sequence then it will be reproducible.