Not so much for deciding what to serve to a request, but moreso to analyze your audience and figure out from a technical perspective what is feasible and what is not. For example, a recent decision I made was to decide whether or not to rewrite a portion of our application in a javascript framework that did not offer IE8 support. I was able to quickly validate what portion of my userbase I was affecting by looking at user agent strings.
Request headers are a fantastic way to capture this little bit of information. Using javascript feature checking depends on a javascript engine successfully loading and parsing whatever scripts I've provided to the client. Request headers are simple plaintext that don't fail in even the most bare bones of situations.
No need to do browser detection. Just make your site work without JS as well :)
Nothing ever guaranteed the user agent header would be consistent or present. Any re-use of a user-agent string adds bits of data that can be used for tracking.
Have a css media query for every possible height, and every possible width, that loads a 1px empty png.
Amen.
I agree with the commenter below who suggests we use random strings instead. Unless there is some sort of published standard for what this header should contain and what each byte means, the contents may as well be random.
The "history" of "user-agents" (about 20 years) is not very long, assuming the web is going to last more than 20 years into the future. Taking a long view, we are still in the nascent phase.