You should be able to detect if the stream is seekable (by checking the result of 'lseek(fd,0,SEEK_CUR)') and only buffer if it's not.

Of course, if you're really paranoid, the file could get changed out from under you. But honestly you're probably screwed either way with an attacker who can do that.

Loading everything into memory at once shouldn't be necessary to produce a hash of the entire input. All of the hash functions currently supported allow for incremental hashing. That means you can hash in blocks instead of all at once.

The input still needs to be cached for eventual output in case the hash matches I assume.

More importantly, so nothing is output if the hash doesn't match.

In that case, you could still cache if you want but at least you now have the option to manage your memory instead of always crashing with large input

Another fun option would be to use a tree hash. The distributor of the content-to-be-verified would then envelope it in a format (to-be-defined) that includes proofs-up-to-root every N bytes. Then the verifier could stream, and know that everything it emits fits into the target hash, needing only N bytes of working space.

Caveats:

The source doing-the-enveloping will need two passes (and enough working space for the remainder-tree).

An attacker could still choose the moment-when-content-goes-bad; in the envisioned use of immediately-executing the verifier output, this might leave things in a problematic/resource-consumptive state. (Scripts could be hardened against such partial-execution failures.)