The delegate obviously breaks down when you have more than one method. For that interfaces work just fine. Even in the case of using a delegate where you capture variables, the C# compiler creates a new type, instantiates it and passes the method in place of the delegate. It's hardly any different to deriving from an interface.
I just tend to use interfaces more out of habit even for single methods. Creating an object expression in F# is pretty much the same outcome as creating a closure in C#.
EDIT: I guess the other problem is that F# functions are not directly compatible with Func<_> and Action<_> types, so when calling a C# function expecting these, you need to wrap the call in an anonymous function. C# interfaces and F# classes are compatible though.