Watch the beginning of Totoro to get an idea of what an abandoned house in rural Japan looks like (rundown, in the middle of nowhere). Vacation homes anywhere get quite dusty in a mere week, so you'd be looking at a spring cleaning event on every trip (very much like in the movie).
Bear in mind also that flight tickets to Japan aren't exactly cheap either; you can get a week worth of hotel stays in the US for the price of the round trip flight from US to Japan.
It's not like the "spring cleaning" mentioned by others is a deal breaker either, it's probably possible to find someone to do some of that for pay if you speak enough Japanese.
Obviously having something close enough to trains / relevant cities might be a long shot, but the payoff would be huge if anything like this existed.
Fingers crossed that someone with insight responds
In their case, they had fairly specific location requirements and thus restricted themselves to Ibaraki prefecture. So fairly close to Tokyo (1 hour train ride I believe?), but they paid much more than $500 -- more like $20-30k if I recall. Once you go through the bureaucracy of buying it (making sure it doesn't have a tax lien, etc.), you're basically faced with a falling-down hoarder house that you have to clean out and likely tear down and rebuild completely. Theirs was also a quite large lot and a nice traditional Japanese farmhouse called a minka or kominka (https://en.wikipedia.org/wiki/Minka). So, not typical of these $500 houses -- those are probably going to be further out from the cities, much smaller lots, and complete teardowns. But do expect to invest significant time and money to get it to a habitable state. And you'll likely need to speak much more than tourist Japanese to deal with the local bureaucracy and builders.
But if you're willing to put in the effort, and you do like the quiet Japanese country life as a getaway, it could well be a good investment. This is pretty much my plan for (semi-)retirement in a few years.
Can I, as a citizen/resident of the US, buy one of these $500 houses and keep it as a vacation home for whenever I feel like going to Japan and chilling?
That's literally less than 3 days worth of hotel stays at a decent hotel, and it sounds like an excellent deal if there aren't other strings attached.