As an example, xxHash [1] would probably get you an easy 5-10x performance improvement over Blake2b. So there are some easy improvements if the cryptographic requirement is not needed.

[1] http://cyan4973.github.io/xxHash/