Is there not any scope for improvement on this front using alternative HTTP auth schemes though? (As far as I'm aware the standard has room for schemes other than Basic and Digest, although I suppose limited to the challenge-response format)
At any rate, it'd be nice if there was an effort to work with the HTTP standard (or if not possible, propose extensions to it) and aim for clean protocol design where at all possible when driving to improve API security.
SSL + HTTP Auth has been pretty good for us, and it's a good fit for the RESTful design goals of the protocol, so I'd be sad to see it thrown out, rather than improved where necessary.
If there's a single advantage to conveying a username/password in a dedicated HTTP header as opposed to any other way of using HTTP to convey the same information, it hasn't been well articulated to me. Down with HTTP Authentication!
The others: Host - necessary for multiplexing over the predominant transport protocol Date - necessary for caching Referer - mostly useless and optional anyway User-Agent - information about the client and essentially useless Server - useless
And then there are the bastard children: Cookie/Set-Cookie - used to turn HTTP into a stateful protocol Authorization/WWW-Authenticate - used to transmit information about the relationship between the client and server.
The statelessness of HTTP is one of its major strong points, making it easy to load-balance and scale, more easily compatible with a wide range of middleware, etc. The redundancy involved in repeating certain headers is a design trade-off made for a reason.
There's a bunch of trade-offs here in the protocol design though, evidently. Coming at it from a security perspective seemingly leads to a different view of those tradeoffs than (say) Fielding has.
If you have an application with a really pressing need for stateful session-level protocol features though -- perhaps HTTP isn't the best choice anyway?
In a browser context, most of the UI criticisms have little to do with the protocol itself and could be easily addressed if browsers would add a little more HTML + javascript API support for doing HTTP auth logins and logouts. Admittedly that looks unlikely to happen in the near future, which is a shame IMO. IIRC they were considering it at one point for HTML5.