Re:Announcing Rust 1.7
by wyldfire on Thursday March 03, 2016 @05:16PM
(#11220488)
Does Crater ever find unstable tests that don't pass consistently on a good baseline? If so, do you just blacklist those packages and/or feedback improvements?
NM: after reading more carefully, it's clear that at least in the case where rust causes the "regression" you feedback the improvements, so likely the same for unstable tests.
Re:Announcing Rust 1.7
by nightpool on Thursday March 03, 2016 @08:50PM
(#11221509)
Well, the "tests" here aren't actually tests—they're whether the package compiles or not. If the compiler is being inconsistent, that's a pretty big bug ;)
Re:Announcing Rust 1.7
by wyldfire on Friday March 04, 2016 @10:07AM
(#11224174)
Ah, I thought it was doing "cargo test" to see if the compiled code had a functional regression. But that would be challenging to investigate regressions (especially in the face of unstable tests).
NM: after reading more carefully, it's clear that at least in the case where rust causes the "regression" you feedback the improvements, so likely the same for unstable tests.