And hardware, they actually deploy to hardware they buy locally from retailers to verify things still work too last I checked. Because there is always that "one popular laptop" that has stupid quirks. I know they try to focus on a spectrum of commonly used models based on the telemetry too.
And in case of closed source software, that isn't publicly available, crates wouldn't work.
Also I think there are magnitudes more C libs/apps, than Rust crates.
But what that doesn't solve is making sure that the resulting builds actually works. Cargo, for Rust, makes running some form of tests relatively easy, and Rust is new enough that virtually every published package is going to contain some amount of unit tests. But for random open-source packages? Not really. Pick a random numerics library--for something like an linear programming solver, this is the most comprehensive automated test suite I've seen: https://github.com/coin-or/Clp/tree/master/test
Huh? I don't follow. There are more libs to test and they aren't standardized. How does that mean theoretical Crater will take less resources?
Did you mean excluding non-testable code? That doesn't prevent future glibc-EAC incompatibility.
I agree it's embarrassingly parallel, but the expenses scale almost linearly.
Overall I agree, for MSFT it's doable, but I doubt any Linux distro has enough money to continually provide this level of support.
The Linux Kernel does have such a project doing batteries of tests. Userspace may not, but that's not a "unit test" problem. In fact it's the opposite, it's integration tests.
> Linux is really hurt here by the total lack of any unit testing or UI scripting standards.
> standards
I've been very impressed reading how the Rust developers handle this. They have a tool called crater[1], which runs regression tests for the compiler against all Rust code ever released on crates.io or GitHub. Every front-facing change that is even slightly risky must pass a crater run.
https://github.com/rust-lang/crater
Surely Microsoft has internal tools for Windows that do the same thing: run a battery of tests across popular apps and make sure changes in the OS don't break any user apps.
Where's the similar test harness for Linux you can run that tests hundreds of popular apps across Wayland/X11 and Gnome/KDE/XFCE and makes sure everything still works?