The point is that Shinkansen never offers discounts. The price is always the same. So if you are poor or if your company asks you to travel for business on the cheap, they will save on the trips using airlines instead of trains, because there is virtually no difference between ANA and any other discount airline for such short distances.
> That said, flying takes 1h30, plus an hour on each end, give or take, and then you have to get to the city
The Shinkansen arrived either in Shinagawa or Tokyo, which may be far from where you want to go as well. Let's not assume that "in the city" is close to your actual destination. Most of the time I need to take subway/train for another 30-40 mins so I'd say it's about the same with the plane. And security checks in Japan for domestic travel is very, very fast.
But like I said: trains don't have security checks, or the 2+ hours of boarding/taxiing/waiting that come with any commercial flight. A three-hour train ride is...three hours. A three-hour flight will take half of your day (at least), and then you have to get from the airport to your actual destination.
For a journey from Tokyo to Osaka, I'd have to be pretty cash-strapped to prefer a flight simply because it was $50 cheaper than the train.
That said, flying takes 1h30, plus an hour on each end, give or take, and then you have to get to the city. The train takes 170 minutes (~3 hours), doesn't require security, and drops you right in the center of town. That seems pretty competitive to me -- which is probably why most trains I've taken in Japan have been at least half full, and popular routes sell out on a regular basis.
I won't dispute that Japan has spent a great deal on their train system. The US has easily spent hundreds of billions on the highway system. It's a matter of priorities.