If a single test takes a few hundred milliseconds then that means you can only run a dozen or so such tests before you hit that problem, which isn't very many. I'd say that a few hundred milliseconds is quite slow in this context.
I don't think that's a problem at all. I usually run one functional test at a time while developing (ideally runtime < 30 seconds) which covers the code that I'm changing.
90% of the bugs that I introduce accidentally will get picked up by that test while I'm refactoring.
In the background (often on a different machine) I will have the full set of tests running constantly. Those might take hours to complete (sometimes a whole night). They will pick up the 5-10% of bugs that slipped through the net eventually, though - which is ok.
This is way better than having a unit test suite that doesn't even come close to mimicing real life usage, and is a PITA to maintain because it's naturally tightly coupled to your architecture (this bites you back when you refactor).
>If a single test takes a few hundred milliseconds then that means you can only run a dozen or so such tests before you hit that problem, which isn't very many. I'd say that a few hundred milliseconds is quite slow in this context.
One 500 millisecond test for this particular use case will pick up 90% of bugs. No, that is not going to interrupt your development cycle.
Yes, a full suite of tests that fires up a VM for every different possible kind of filesystem will take hours. That's perfectly ok. You can go get a coffee while it runs.
Sometimes it's just not practical to impose a hard limit of "a few seconds at most" on your test suite, especially for larger projects. (some projects have test suites that take hours to run, and simply cannot be any quicker unless they were less thorough).
It's not supposed to be easy. Doing it well is really hard and way underappreciated.
>Testing side-effecting code is hard. This is well established. It's also convoluted, complex, generally brittle.
It's not necessarily brittle when done right but it's hard, it's convoluted and it's complex.
>Before the test, create the input AND go to the filesystem, prepare the input and the spot where output is expected. After the test, check the output AND go to the filesystem, read the files from there and check their contents. Everything is intertwined: the prep, the implementation of the code under test, and the checks at the end. It's specific to my filesystem. And it's slow. No way can I run more than a few of these each build.
This doesn't sound like it would take more than a few hundred milliseconds. That's not slow.
Furthermore, you could definitely run it on multiple filesystems by firing up a virtual machine for each filesystem and running the test. That would be slow, but you don't need to do it often.