News for hackers. Stuff that matters.
Re:Testable IO in Haskell by chriswarbo on Sunday June 21, 2015 @05:33AM (#9752774)
> For example, suppose you want to test a scheduler, or any mechanism that places a timeout on an operation. You could abstract away the entire timing mechanism, but that is exactly what you're trying to test: maybe the developer used some system function wrong, and is now waiting 5 seconds instead of 5 milliseconds?

Time is an IO effect, so it's true that mocking it away removes that test. I'd treat that kind of "do I understand the API?" issue as an integration test, and split the problem up a little:

- "Mock" the timeout calls, to allow as many tests as possible to run outside IO

- Write tests where the operation always succeeds (eg. "Just x")

- Write tests where the operation always times out (eg. "Nothing")

- Write tests which do a combination of success/timeout and a bunch of different interleavings (QuickCheck would probably be useful here)

- Write some small integration tests, eg. give a timeout of "5" to a process that sleeps for 5 seconds, it should time out; give a timeout of "5" to a process that sleeps for 4 seconds, it should succeed; and so on.

> Mocking (or IO isolating as you call it in Haskell) ends up producing really good unit-tests, but often the more important tests span more than one small unit, and making those non-flaky is very hard.

Haskell's IO isolation causes the vast majority of a program to be defined outside IO. You're right that the trickiest parts are often in IO, but at least those are pretty much separate from any complex calculation/logic.

Re:Testable IO in Haskell by pron on Sunday June 21, 2015 @06:39AM (#9752894)
> but at least those are pretty much separate from any complex calculation/logic.

Yes, but in imperative languages (with meta-programming) you can disentangle mixed IO and logic even without changing the code (or if it's in a third-party library). The advantage is that it allows you (not always, but sometimes) to support even larger tests in this way.

Getting those tiny, one-function unit-tests to run predictably is not hard in any language; you just make them smaller and smaller and mock everything else. The tricky bit is making larger tests run predictably, and a flexible, hackable runtime has the advantage there. That's precisely where meta-programming shines.

Re:Testable IO in Haskell by chriswarbo on Sunday June 21, 2015 @08:51AM (#9753216)
> in imperative languages (with meta-programming) you can disentangle mixed IO and logic even without changing the code

I don't understand what you mean by "mixed", "disentangle" and "change".

I would say "mixed IO and logic" is when one block of code describes both how to calculate some value and which side-effects to perform. "Disentangling" these would mean separating the description of the calculation from the description of the side-effects. Doing that "without changing the code" doesn't make sense to me.

Re:Testable IO in Haskell by pron on Sunday June 21, 2015 @09:14AM (#9753300)
What I mean is that if your code (or a library you call) mixes logic with IO, meta-programming lets you capture the IO calls in a mock (like turning it into an IO monad), which you can then query and verify, without changing the code and the API's it's using.

E.g. on the JVM you do that with Mockito (similar mocking libraries exist for other languages/runtimes). Say you have a (silly) method that does computation and IO:

    void printDouble(int x, PrintStream out) {
        out.println("Result: " + (x * 2));
    }
You can then do:

    @Test
    public void testPrintDouble() {
        PrintStream out = mock(PrintStream.class);
        
        printDouble(3, out);

        verify(out).println("Result: 6");
    }
No output is actually done. The call to println (which may be deep in some library printDouble calls rather than directly in the tested code). We've essentially replaced any object that does direct IO with something akin to an IO monad.

We can query the mock in more sophisticated ways, too. Here's a very contrived example:

    void printDoubleWithLog(int x, PrintStream out, PrintStream log) {
        log.println("Computing...");
        out.println("Result: " + (x * 2));
        log.println("Done.");
    }

    @Test
    public void testPrintDoubleWithLog() {
        PrintStream out = mock(PrintStream.class);
        PrintStream log = mock(PrintStream.class);
        
        printDouble(3, out, log);

        InOrder ordered = inOrder(out, log);
        ordered.verify(log).println("Computing...");
        ordered.verify(out).println("Result: 6");
        ordered.verify(log).println("Done.");
    }
This contrived example also verifies the relative ordering of IO done on two separate streams. Again, no API change is necessary, and the actual IO calls may be deep in some library.
Re:Testable IO in Haskell by chriswarbo on Sunday June 21, 2015 @01:09PM (#9754102)
OK, so you're on about a physical/temporal separation of IO and logic: the logical calculations can be performed separately to the IO actions, and vice versa.

I view that as pretty trivial though; it's just a case of switching out the language's IO primitives, which are effectively free variables. With most "large" languages (Java, Haskell, etc.), the implementations are usually gnarly enough to make such substitution a significant engineering accomplishment, but that's an artefact of the situation, not the approach. As an analogy, the task of fitting corrective optics to the Hubble space telescope was only difficult because it was the Hubble space telescope; fitting adaptive optics is trivial enough that bespectacled children do it every day.

In light of this, I think that disentangling mixed-up definitions is a much more important and interesting problem. It's non-trivial to extract pure computations from imperative code (whether they're written in Java, or Haskell, or whatever), yet there is a great potential benefit for improving code, in terms of understandability, modularity, re-use, maintainence, etc. The fact that pure code is easier to test, or that side-effects can be substituted for the purposes of tests, is really a minor point in comparison. They are workarounds for running code which has too many responsibilities. Much better to avoid the problem in the first place by writing separate code for separate concerns.

Re:Testable IO in Haskell by pron on Sunday June 21, 2015 @02:01PM (#9754314)
Yes, I agree that it's better to write good code than bad code :)

Clear separation of concerns is an important aspect of good code, and every language teaches this practice. I don't think Haskell has any advantage there.

I also think -- and this is probably a point of contention -- that the way Haskell separates pure computation from side-effects is a bit arbitrary. Haskell defines computation in a way similar to lambda calculus, which designates as impure any change to something which may then affect a function's result not through its arguments. In short, it equates the mathematical notion of the function with the software notion of the subroutine. Haskell subroutines are mathematical functions, period (well, except for unsafe etc.). I don't think that this is the best way to describe code, or to define separation of concerns (It also makes Haskell jump through hoops when it deals with things that are natural for computation but don't fit well with the notion of a mathematical function). A good separation of concerns in Haskell is not necessarily the best organization for Java/Python.

Re:Testable IO in Haskell by simplify on Sunday June 21, 2015 @05:21PM (#9755034)
What do you think would be a better way to enforce this separation of concerns? (genuinely curious)
Re:Testable IO in Haskell by pron on Monday June 22, 2015 @06:56AM (#9757427)
I don't know, other than good supervision by an experienced developer :) This one, though, isn't that good, IMO. Perhaps future effect systems will show us the way.