Are you imaging some form of refund on the billions in infrastructure spending? Japan has nothing to gain by canceling and freeing the athletes and IOC from their responsibilities. Why shouldn't Japan get their own private Olympics? They paid for it.
Japan has not paid 100% of the costs already. Cancelling will avoid throwing good money after bad.
If we are _only_ thinking about finances then the question is if Japan will make more money from hosting the reduced olympics than it would cost to finish all work required to host it.
It would be possible to just vaccinate every athlete (talking less than 10k people). It might be possible vaccinate all of the staff as well. But in a country where not even all the doctors have gotten vaccinated yet, it's not a great look, politically.
And if you choose not to do mass vaccinations, you're talking about doing a mass event with huge risks of spread at every level.
In all seriousness though I think Japan has plenty to gain by acting as a global leader, it’s not all about money, and even if it were all about money then unfortunately sometimes cutting loses is the best you can do...and no one really knows how it might work out legally/financially, there may be refunds, or more accurately legal remedies for many of the contracts and investments, and there could be untold amounts of insurance coverage.
You are forgetting the most important constituency driving this: "Senior Japanese politicians also in on the take."