This is not technically true. You can throw exceptions from pure code, either explicitly or with something like an incomplete function definition (which the compiler should warn you about). However, it is considered very bad practice to do so.
Catching "undefined" from pure code is the danger that leads to massively confusing semantics—it breaks down the "value" concept badly. The `spoon` library is a great example of this and it's scary to see it in places. That said, it's not terrible for wrapping up foreign code that isn't quite unsafe enough to need "IO" treatment.
The best usage of "error" is to mark completely impossible situations. These show up easily when you do dependently typed stuff with GADTs, but can also exist due to various algorithm invariants.
People tend to avoid this behavior except in rare circumstances. That said, it's easy to replicate locally with the Cont monad. These local reflows are easier to reason about as well. You can even build it atop delimited continuations for more control. These are great for covering early stopping in searches, for instance.