ffseika0304

Links:

- Official site (bilingual, EN: https://seika.ltd/en/ | 中文: https://seika.ltd/) — includes the live quota command center numbers behind the "8 incident rules" story - Source: https://github.com/ffseika0304/shanhai-gateway - README (EN): https://github.com/ffseika0304/shanhai-gateway/blob/master/R...

The storage-vs-egress split is where we landed after learning this the hard way. On the storage side we mask aggressively before anything is written (keys, intranet paths, phone numbers - better over-masked than leaky). On the egress side, raw bytes go upstream, and making egress masking opt-in per backend is the line we're moving to: untrusted upstreams get scrubbed, trusted paid backends get raw.

The reason egress masking isn't just "do it everywhere" is coding traffic. A phone number inside a test fixture, an intranet IP inside a config file being edited - mask those and the model faithfully produces wrong edits. So the egress pattern set has to be narrower than the storage set (identifiers yes, code-shaped content no), and the switch has to be per backend, because "how much do you trust this upstream" is not one answer across a pool.

Curious how you draw that line for coding agents - is sensitive-data traffic ever routed through coding workloads in your deployments, or do you treat those as separate compliance domains?

Operating a Go gateway in production (single machine, ~20B tokens/month across several upstreams), usage-reporting quirks are exactly where our pain concentrated too. What survived for us:

- Treat every upstream try as an attempt with its own usage record, and let request cost be whatever the attempts sum to. Without attempt-level accounting, retries make your per-request numbers lie: you count the first attempt's tokens but pay for the retry. - Vendor-reported usage is advisory, not gospel. Some providers emit usage only on the final SSE chunk, some want stream_options, some never report it for streams at all. The parser zoo is unavoidable, but the damage is contained if the rest of the system treats usage as best-effort telemetry. - The sneaky case is disconnects, not usage: a client disconnect storm once looked identical to backend failures and put every healthy backend into cooldown until the pool was empty. Disconnects are network, not backend health - classifying them separately bought us more than any routing policy change.

How does Respan handle the retry-accounting side - do plugins see per-attempt records, or is usage aggregated at the request level after middleware runs?