To avoid regression I have some simple code examples I compile and execute, and I compare their output to "known good" versions.
I reached a point where I wanted to write a "sort array" routine and my immediate thought was to generate an array of 50 random numbers, sort them, and print them. But of course that wouldn't give me predictable output for my test-driver.
In the end I decided I'd do that when run interactively, but for testing purposes I'd just sort the characters in a string "The quick brown fox .." and while it isn't super-convincing it's enough to let me see regressions in my sorting function and/or array indexing runtime code.
But ideally sort is something you want to test with something like quickcheck/hypothesis, not gold tests (and I say that as probably the world's number 1 proponent of gold tests).
It’s not elegant but I don’t see any better way to do it to maintain backwards and forwards compatibility.
My plan is to control engine versions via a manifest.json or similar that points to applicable engine versions as a compatibility layer
On the other hand, it's STUPENDOUSLY useful to have "default" random functionality in your core library, for the "just give me a random number" or "shuffle this array, I don't care how" users, who don't really care about the details. But if you do that: always seed it with some external entropy (current time or /dev/random or whatever), don't even allow users to seed it. That means you can improve it in the future, because users already can't ever rely on the sequence. If the users do want to rely on the sequence, they should have to specify the exact engine they want.
TL;DR: System.Random in C# should not ever have been seedable, big mistake.