Second this. Even some cheap (~$30 per night) guesthouses have onsen. Also you can often get into the fancy hotel onsen by paying a small fee, even when you are not staying.