You're making the completely unsupported assumption that the average internet user understands domain names. Like patio11 suggested, if a user from a Japanese IP is accessing google.com, it's likely that this address was pre-configured in some way: the search bar was set up like that, or the company sysadmin set up the home page like that. But even if the user manually typed "google.com", if they're in Japan it's still (way) more likely that they wanted a Japanese version rather than the English one.
Also, they either use the English version of the OS and browser, or the one in their language, and both come with the default correct preferred language settings.
The same goes for company systems. You don't give someone who can't handle English an English language system, and the idea of setting up a system in language X but pre-configuring that in some what that English becomes the preferred browser language is a really odd edge case.
Even if such scenarios are more common, it still doesn't excuse telling people that make explicit choices to go fuck themselves, especially in multi-lingual countries where such explicit choices are common.
Japan may be relatively homogenous, but most countries in Europe are either formally multilingual or have large immigrant populations. Many thousands of people are regularly being confronted with misplaced assumptions about their preferred language.
You can't assume language from geo-IP databases, it is the most unreliable and inappropriate method imaginable, and you are 100% sure to regularly piss people off.
People don't know the difference between the google search bar and the address bar.
Understatement.
Also, if in Japan, I would be surprised to find google.com not to be in Japanese, and I would expect to have to perhaps make some kind of preparations for the fact that the majority of the world does not speak my language and that my electronic devices and services are not universal metatrons run by omnipresent deitys either.
If you are a local you are probably very used to your country domain, it is probably on par with .com.
Now the Accept-Language is quite subtle and guess how that is set up in an internet café? Can you even change it? Will the next customer be happy about that? I'd be mighty impressed if more than 1% of the people that surfs the web has a clue about Accept-Language.
Also, if one asks for a local country domain that really should be the localized version, sometimes I actually want to use google for a language different other than my own.
If I for instance wanted to search in English on something that has a similar wording as Swedish I'd like to go to google.com in order to quickly get rid of the Swedish-bias (since that doesn't work I sometimes use google.co.uk instead).
Many people are however, and mass media, which includes Google, however they may try to deny it, lives by the numbers, which means adapting to the lowest common denominator. Just think, as the Internet spreads, Google will evolve (devolve) closer and closer to broadcast TV!
Google is making the (completely wrong) assumption that those people cannot type google.co.jp instead of google.com and make their own choices.
But at least in Japan they're probably right about the language. Google pulls the same crap in multilingual countries, and gets it wrong a lot of the time. This isn't Google being smart, this is Google being lazy and insensitive.