SRV is definitely the ideal record. In this case I can see why a pure HTTP approach was chosen (it avoids browser dependancies) and it irks me that developers of web-based tools tend to develop inside the HTTP bubble rather than using the broad and highly capable infrastructure of the Internet, because this kind of constraining outcome is the result.
SRV would be an excellent choice for HTTP/2.0 as well, rescuing us all from apex quasi-CNAME hacks and the like.
However in my initial remark you can substitute in the use of "a well-known subdomain"[1] for "SRV record" and still achieve a better separation of concerns than simply using the RHS of the identity string.
[1] c.f. DKIM's slightly hackish but effective use of an intermediate underscore to denote infrastructure DNS entries. Won't work for A records though IIRC.