What you do with it (or, don't do with it) is not morally neutral. I believe that hoarding it, preventing others from living in or purchasing it, is immoral.
Buying a house in order to leave it vacant is morally neutral. To wit,
Mike Murderer buys up real estate along the river and leaves it empty so that the dead bodies he stores inside are not discovered. Mike has acted immorally!
Mayor Susan convinces the city council to buy up real estate along the river and leave it empty to facilitate moving the residents there to higher ground to avoid expensive and dangerous seasonal flooding. Susan has acted morally.
But they both took the same positive action: purchasing real estate. This action was morally neutral.
As other people in this thread have correctly noted, it’s the justification not the action that has moral implications.