I'm arguing it's better to build the duck typing features that foreach and initializers use into the language itself. Every hard-coded keyword and to some extent, feature, is a strike against the language. foreach and stuff like that should be part of the stdlib.
But then you have to make the language and syntax flexible enough to deal with that – while also retaining ease of use, compilation speed (you don't want to go into C++ territory there) and perhaps other factors. All in all it's probably a tradeoff and regardless which way it sways, people are inconvenienced by it.
And arguably there are other languages which can and do fix that gap. Nemerle comes to mind. .NET allows you to mix languages freely so I think switching for parts where it makes sense isn't such a bad idea.
Now you can argue whether it is better to just map to the methods Add<T>(T item), GetEnumerator<T>() and Dispose() or if you require an interface like IInitializable<T>, IEnumerable<T> and IDisposable. On one hand requiring an interface just adds some overhead, on the other hand it makes the intention much more explicit. They came up with different solutions for different cases - using requires the interface, foreach supports both ways and there is no interface for collection initializers.
I don't understand why foreach supports both ways - if I implement GetEnumerator() adding IEnumerable to the interface list seems no unreasonable requirement just like in the IDisposable case. I can understand that there is no interface for collection initializers because this feature was added in C# 3.0 and would have required adding an new interface to all existing collection classes, or changing interfaces or starting with an inconsistent implementation. Would be interesting to hear about the details of the decisions.