Their only reason to exist is to support a use case of a 1:1 relationship between classes and functional units (or methods and functional units, or parameters and functional units, etc.), which is almost always an 80% solution that makes the other 20% extremely hard and/or impossible. I would go so far as to say that every single use of an attribute is your own code begging you for a better design. Unfortunately you're sometimes stuck with them, but that's only because you're sometimes stuck with a framework that itself is begging its creators for a better design. I wonder if Attributes were never added to the language, if developers would have just made those better choices to begin with. From my vantage point, they were a clear mistake in a language that was otherwise very well designed.
They certainly are nothing like the ability to attach methods to functions. A better example of that kind of convenience in C# is extension methods, which you can certainly define on types like Func<T>. I love extension methods and miss them in every language that doesn't have them!
In .net methods, properties, member variables, classes, etc. can have attached attributes that are objects. Attributes can be interrogated at runtime using reflection.