Operating a Go gateway in production (single machine, ~20B tokens/month across several upstreams), usage-reporting quirks are exactly where our pain concentrated too. What survived for us:

- Treat every upstream try as an attempt with its own usage record, and let request cost be whatever the attempts sum to. Without attempt-level accounting, retries make your per-request numbers lie: you count the first attempt's tokens but pay for the retry. - Vendor-reported usage is advisory, not gospel. Some providers emit usage only on the final SSE chunk, some want stream_options, some never report it for streams at all. The parser zoo is unavoidable, but the damage is contained if the rest of the system treats usage as best-effort telemetry. - The sneaky case is disconnects, not usage: a client disconnect storm once looked identical to backend failures and put every healthy backend into cooldown until the pool was empty. Disconnects are network, not backend health - classifying them separately bought us more than any routing policy change.

How does Respan handle the retry-accounting side - do plugins see per-attempt records, or is usage aggregated at the request level after middleware runs?