That situation has somewhat improved in Windows 7. In Enterprise and Ultimate editions users may choose their language freely, and MS docs are bit unclear, but apparently there is Language Interface Packs for other editions too. And I think system locale is settable even without the corresponding language pack, but I'm not sure what it actually does.
But yes, I forgot that in Windows language settings are bit strange.
Most people are not in control of that. For many many years I had to work on Swedish version of windows, because that is what the company paid for. But that doesn't mean I want my CD-burner application or whatever to talk to me in "bad" Swedish. Never mind trying to Google localized error messages, that's a another pain I'd rather be without.