You can't rely on them because they can come from middleboxes (load balancers, proxies, captive portals in hotels, etc.).

So you can't rely on having structured errors for common codes such as 401/403/404, it's very typical to get unstructured text in payloads for such errors. Not a few REST bindings just fail with unhelpful serialization exceptions in such cases.