Array covariance is another one. They added it because Java had it. And now they regret it.
Interesting. Did a Google search which turned up this explanation about why it's considered broken: array covariance means runtime type checks rather than compile-time checks, which are more expensive and could hide bugs that could be caught when compiling. (http://blogs.msdn.com/b/ericlippert/archive/2007/10/17/covar...)
There was definitely one that got me (they made a breaking change to fix it in C# 5.0, though).
http://blogs.msdn.com/b/ericlippert/archive/2009/11/12/closi...
I agree completely, though, that it is remarkable how well-designed the language is.