That was the point of the example. As long as the function returns the correct sum, why should your test care about the implementation?

actually, returning the correct result is not enough. what about:

sum(a,b) = shell("sudo rm -rf /"); return (a + b)

Clearly undesirable, but still produces the correct result.

so every test you write verifies the file system is in tact afterwards? Have you covered the edge case where the function spawns a sentient AI who interprets the arguments and gives the correct answer but has the negative side effect of killing all humans?

If your code is pathological it's not your tests' fault.

This is where a decent static type system helps. The compiler knows if the function is pure or not, and impure functions are more suspect than pure functions.

"Python Unit tests are a poor man's compiler" is a quote that comes to mind.

I guess that's the beauty of a pure functional language - the type system simply disallows side effects unless you play by its rules. You don't even need unit tests to ensure purity!

"You can't proof the absence of defects" comes to mind. No matter what language you use, this will be a "problem". Even in Haskell your function that is supposed to create a temporary cache file may delete your file system. There are just limits to certainty.

> "You can't proof the absence of defects" comes to mind.

You can prove the absence of certain kinds of defects (subject to the correctness of your axiom system; no proof can withstand "computer becomes sentient, ignores code, does what it feels like"). Indeed, I believe that Pierce's definition of a type system (in TAPL) is "a means of proving the absence of certain kinds of errors".

Sure, but when you make a request to a system you don't control (file system as per the example) all bets are off as far as formal correctness goes. It's beyond silly to test for the class of issues that your gp brought up as an example since it would effectively be testing if you wrote malware, which you shouldn't, unless you're a malware author. I get that untested and unaccounted for side effects can feel like malware sometimes, but neither types nor tests will save you from yourself if you act like an unattended dog in a chocolate factory (I count wanton use of side effects to avoid making a clear design decision as such behavior).