Re:Show HN: Arch – an intelligent prompt gateway built on Envoy
by hahdflakdfwdasd on Wednesday October 16, 2024 @06:18PM
(#41864396)
Since its an envoy filter it was trivial to setup in Istio Ingress. Good stuff.
[ Parent
| on HN ]
Re:Show HN: Arch – an intelligent prompt gateway built on Envoy
by adilhafeez on Wednesday October 16, 2024 @06:41PM
(#41864596)
You are right, since arch is an ingress wasm filter it can be setup inside Istio just like any other envoy filter. You would need to pass arch_config someone which should be easy. We will have samples/demos for Istio and K8s deployments sometime in future. If you want us to focus on this area more please go ahead and create an issue in our issues page at https://github.com/katanemo/arch/issues
Re:Show HN: Arch – an intelligent prompt gateway built on Envoy
by sparacha on Wednesday October 16, 2024 @06:23PM
(#41864438)
Great to hear that - walk us a bit more how you are planning on using the filter? The filter today does communicate with a model_server service for the intelligence piece. Were you thinking of using the filter for outbound LLM traffic management?