Your client sounds like an insensitive scumbag, and you sound like a pushover.

I suggest you squeeze hard for the money owed to you and tell him you're going to keep the site.

Is he going to like you? No. Will he at least have a shred of respect for you? Yes.