Any old working flip or bar phone. 911 calls don't require a plan or service contract. You'll only be able to make 911 calls with that, however. If you want something that will let you make ordinary calls or text, you'll need something like a prepaid phone, which are quite inexpensive as well.

All the prepaid phones I've found seem to have minutes that expire. That doesn't seem ideal for something I'd never use.

There are plenty of cheap candybar phones that last more than a week on a single charge. I use a Wiko Lubi, it cost me around $25 and does calls and SMS just fine.

How much are the ongoing costs though?

I live in Europe, so it's probably not comparable. Around here you can get a prepaid plan with no mandatory payments, the only requirement being making a paid call every once in a while to keep it active.

T-Mobile seems to have a prepaid plan for $3/month, it's still expensive but not that bad. Alternative, you can just get the phone, then switch the SIM cards if your main phone is broken (doesn't work if you lose it instead, though).