Alright, so I finally went and checked, and (according to merriam webster), the following definitions of "bank" are reasonably commonly used (none marked with archaic or obscure).

> a supply of something held in reserve

> a place where something is held available (especially : a depot for the collection and storage of a biological product)

Both of which fit food bank, seed bank, and sperm bank.

Which means that a sperm bank probably is properly a bank, although used without qualification, most people probably mean money bank (or are being intentionally ambiguous to avoid the stigma of needing a food bank/going to a sperm bank).