I think part of the problem is error reporting is still somewhat shaky. To a certain extent you only actually need to handle errors that occur in the real world. Some languages like Java and Python can get you a reasonable amount of information when an exception occurs, but you still have to find mechanisms to report it back, and chances are you won't have detailed program state or operations leading up to the error. But if you did you could in theory release programs with no error handling as alpha releases (in the original sense, not current marketing usage) and see what shakes out.
> I don’t know what kind of testing you have in mind
Using fdr's example of a cat program, it consists of an open, a sequence of read followed by write and a close. I would like to point the language/environment at that and say "test it" with no further effort. Currently I have to write far more code to test, and then add in instrumentation to force various failures (eg make close return an error). The test code in this example would take considerably longer to develop and could itself have bugs!
OK, so I suppose the next question becomes what “test it” means.
Perhaps you want your test system to make some kinds of automatic deductions about the intended behaviour of your code and then to verify that the actual behaviour matches? If that is the case, what kinds of deductions would you want to be handled automatically for you?
Or maybe you’re thinking of some sort of automatic simulation of possible failure cases when you get to the open/read/write/close operations, serving a similar roles to things like mocks and stubs in unit testing?
Some of these are the kinds of problems I’m hoping a good effect system will help with. If you know that the operations of opening, reading, writing and closing files must occur only in certain orders, you can automatically detect violations of those rules. And if you know all possible types of cause and effect that can be relevant within each part of your code — that is, you can identify all stateful/impure/externally observable results — then you can identify any of them that aren’t handled according to some set of rules. As long as you can figure out what you want those rules to be, that is...
It would be something similar to all possible code paths being executed with the program in all possible states. Heck right now I'd settle for even a small subset of this.
> Or maybe you’re thinking of some sort of automatic simulation of possible failure cases when you get to the open/read/write/close operations, serving a similar roles to things like mocks and stubs in unit testing?
That is closest to it. It is already known that open/read/write/close can fail. They are quite difficult to force to fail, so currently you have to write a lot more code to make that happen. And it gets really tedious once you look at all the combinations. eg open has to fail, open has to succeed and then read fails, open succeeds and the 3rd read fails and on and on. This is for a single ~5 line function! And then if my cat() is made available to a library, how to they test for it failing.
I'm quite happy using supervisory programs to detect rule violations. For example valgrind does an excellent job for memory allocation and usage, and I've never used helgrind but assume it works well too.
OK, I’m following you so far.
Assuming that
(a) our language lets us specify the possible failure modes for each function, and
(b) we have a test tool that can systematically simulate various possible combinations of success and failure,
what would you want to do in each test case?
Put another way, when we run our test tool and our magic replacement I/O functions simulate, say, success on the open and first two reads but then failing on the third read, what happens next? What’s the result we’re looking for to determine whether the test passes or fails?
My test code currently has to put the program into a state where cat can be called, and then has to mock/augment each failure point, and then check the results. Just having the middle piece (mock/augment) done automagically would be a massive help. And then of course cat itself can fail so each of its callers needs a way of testing too. Some hand wavy combination of convention over configuration, documentation and annotations would likely help.
Even the state issue should be automatable in many circumstances. For example you could make a successful run of the program, and then the magic records state at the entrance to code blocks. It can cause each failure circumstance, rewind state to known good and cause the next failure circumstance etc.
I'd even be happy running my code (with no error handling) under some tool that induces the errors - when they aren't handled it asks me what I want to do which typically involves writing code to handle the issue, which it then directly integrates and keeps running until the next error.
Even test pass/fail can be somewhat automatable. The tool records what happens, and then in the future alerts you when there is a difference in behaviour. The response is either that the issue needs to be fixed, or that the new behaviour is correct.
You don’t really seem to want someone who can solve error handling. You seem to want someone who can make it disappear, or at least get as close as possible to that ideal. I feel your pain, but I fear you are asking the impossible. :-(
Some errors are recoverable, and maybe it would be helpful to have default policies like “if the file is read only, have the OS prompt the user to make it writeable and retry” to take care of a lot of the repetitive coding work in those cases. At least then we wouldn’t have to do things like writing a top-level file-handling operation in a loop that tries (in the exception handling sense) to do something, and if any recoverable error occurs, attempts a suitable recovery and goes back to try again. On the other hand, to have some sort of default policy, you’d need something in the semantics of your language to indicate which policies should apply and where. By that point, you might not be doing much better than a try-catch in a loop.
In the end, though, not all errors are recoverable, and that is the fundamental problem. Some errors are fatal, and you just can't carry on as normal or rely on some automatically generated default behaviour, because the semantics of your program are now broken. What would you want to see that you don’t have today in terms of minimising this kind of error handling code or making it easier to test?
I don’t know what kind of testing you have in mind, but one thing I’d really like to see going mainstream is a serious effect system built into the type system of the language. That would be useful for a lot of reasons, one of which might be checking that whatever error handling constructs were available didn’t do things like leaving effects half-applied, transactions open, or resources locked, in the event of a change of plan because of an error condition. I expect some relatively near-future languages will start to pay more attention to formal effect systems as fields like distributed systems, concurrency and security become ever more important in the programming mainstream. But this is a topic far bigger than just error handling...