I guess it just surprises me that having to worry about the state of a stream is a thing I have to do. More, it sounds like I can't really use the typesystem to help here. At least, not from the sound of it.
In praxis a streaming library like pipes, streaming or conduit handles the file handles for you so application code is safe. Safe-ish in the case of streaming but that is a conscious tradeoff between convenience and safety.
You could argue that libraries shouldn't contain unsafe code either and go with a substructural type system, use indexed monads, fake regions or something similar. Whether the extra complexity is worth it is arguable, though.
Returning the stream itself from the "withFile" and trying to run it later still is still a bug.
I like this approach better that conventional Haskell lazy I/O, which I personally find quite confusing.