It's not like a board seat is this magic item that is stuck to you and you can't get rid of of you tried.

If a VC had truly written off the company, and board meetings are not worth their time anymore, they can just resign their seat.