Some of the quirky behavior of the SC1 carrier was even re-implemented into the upcoming SC2 expansion at the behest of fans. Why? Because it made the unit fun.
An examination of what it was that made the Carrier fun might have revealed ways to get similar effects (long range, able to fire while retreating, requires paying minerals for continued use . . .) in a unit that wasn't such a weird gamebreaker. Most people just liked Carriers because it cast a field of enemy unit and Ai stupidity around it. Once players get attached to a unit, you may be stuck keeping it in the game even if it's not really good for overall playability.
Just because it worked with this one unit doesn't mean the overall game experience couldn't have been better with something different in there. But once something is introduced, status quo bias takes hold in a major way.
Edit: I'll concede that you want the AI to be stupid in some ways so that it can be exploited. The Carrier just always seemed to me too extreme.
Actually, it is. Single player is basically a financial dead end: you play it through X times, interact with very few people, and put the game away.
If you end up breaking it in favor of gameplay where players are generating content on your behalf (each other)... the loss rapidly disappears as people stop playing the single-player because it not only sucks, it's irrelevant.
I've noticed that units never seem to have the good sense to target Brood Lords rather than Broodlings when they have a choice. It seemed odd, given that the AI had improved so much in other areas -- pathfinding, running from unwinnable fights, varying its opening strategies. It never occurred to me that things might be that way because it's fun!
Supreme Commander for instance has much better user interface and unit control mechanisms. This makes it much more of a macro game, since micro managing units is not necessary to the same degree. And indeed some people really like micro managing stuff and for the others it serves as an excuse to why they suck at the game.
Although I love Starcraft 2 I think its UI is horrible in comparison to what else is out there. However it is an essential part of the game and it gives it character.
Players get very attached to timing and knowledge based tricks, especially ones that give them an advantage. Does that mean they should get them? There's a tricky balance between alienating new players and rewarding skill and dedication in old players.
For example, take that trick with keeping a couple of void rays charged up on each other. I don't think that's something amazing everyone should aspire to. I think it's obnoxious.
That sort of thing is all right if it gives you interesting decisions to make -- and the video about the carriers argued well enough that it does. It ought to be overt, though: A pre-launch interceptors button, not hidden knowledge that invisible, unselectable interceptors stay out if you keep carriers moving.
Games with matchmaking systems don't need to dumb the game down in order for players to be able to play against each other and each have a roughly even chance of winning.
I'm not a SC1 purist - I'm happy I can rally workers to minerals from a base, and I even like the automining feature at the start of games in HoTS. But removing APM sinks and removing interesting/subtle unit mechanics are two very different things.
The new one's better, but definitely not exploit-free. A dozen speedlings in its mineral line will result in it recalling ALL its forces, rather than just what it needs to deal with the problem.
I sometimes think of the AI as an example of how hard it is to simulate human intelligence.
The Carrier should have been scrapped entirely and replaced with a different air unit. Even to the latest upgrades of Starcraft I, the AI simply couldn't deal with the carrier effectively--they would always attack the interceptors first. This made using the carrier a cheap game-breaking trick against the computer.