Re:Launch HN: Respan (YC W24) – Open-source Rust AI gateway ...
by ShalomDembe on Tuesday October 27, 2026 @10:44PM
(#50268751)
Nice to see an open-source Rust gateway built entirely on plugins. A few areas I’d like to see clarified in the docs or repo, since “everything is a plugin” can easily become complex to manage:
- *Plugin interface/versioning*: How do you ensure compatibility across plugin updates? Is there a defined ABI with semantic versioning or just best-effort contracts?
- *Capability declaration*: How do plugins specify what they support (modalities, streaming, tool-calling formats, limits)? Without this, smart routing risks falling back on ad-hoc heuristics.
- *Fallback behavior and streaming*: What are the exact fallback rules? Do you buffer or stream and then switch? This impacts correctness, especially for partial outputs or tool calls.
- *PII redaction approach*: Are you using structured methods (regex, entity extraction) or LLM rewriting? How do you measure false positives/negatives or avoid introducing new leakage paths?
- *Observability/audit trails*: Do you log detailed metadata for each request—like routing decisions, models used, token counts, redaction actions—to enable proper audit and debugging?
- *Security boundaries*: How do you prevent prompt injection or escalation through plugins or tool payloads? Is there sandboxing or permission controls?
- *Evaluation and testing*: Do you provide a way to replay requests and compare outcomes (cost, latency, safety) for meaningful smart routing evaluation, beyond manual testing?
If these design considerations were documented, it would help teams adopt and contribute with more confidence.
- *Plugin interface/versioning*: How do you ensure compatibility across plugin updates? Is there a defined ABI with semantic versioning or just best-effort contracts? - *Capability declaration*: How do plugins specify what they support (modalities, streaming, tool-calling formats, limits)? Without this, smart routing risks falling back on ad-hoc heuristics. - *Fallback behavior and streaming*: What are the exact fallback rules? Do you buffer or stream and then switch? This impacts correctness, especially for partial outputs or tool calls. - *PII redaction approach*: Are you using structured methods (regex, entity extraction) or LLM rewriting? How do you measure false positives/negatives or avoid introducing new leakage paths? - *Observability/audit trails*: Do you log detailed metadata for each request—like routing decisions, models used, token counts, redaction actions—to enable proper audit and debugging? - *Security boundaries*: How do you prevent prompt injection or escalation through plugins or tool payloads? Is there sandboxing or permission controls? - *Evaluation and testing*: Do you provide a way to replay requests and compare outcomes (cost, latency, safety) for meaningful smart routing evaluation, beyond manual testing?
If these design considerations were documented, it would help teams adopt and contribute with more confidence.