News for hackers. Stuff that matters.

A comment on Show HN: ArchGW – An open-source intelligent proxy server for prompts.

Re:Show HN: ArchGW – An open-source intelligent proxy server... by mikram on Wednesday March 05, 2025 @07:33PM (#43274671)
I think I saw this a few months ago, but never followed up. Why train your own models? Aren't you better off using GPT or something like that to handle the tasks Arch uses specialized models for?
[ Parent | on HN ]
Re:Show HN: ArchGW – An open-source intelligent proxy server... by sparacha on Wednesday March 05, 2025 @07:39PM (#43274719)
Speed. And separately, instruction fine-tuning an LLM for a specialized task like function calling or guardrails == better performance. Even Anthropic and other model providers suggest you separate tasks for LLMs to improve overall user experience

We happen to take those tasks that are non-business or domain specific related and trained our models to offer SOTA performance for 1/10th the cost and 10x the speed. For e.g. Arch-Function can process 5k/tokens per sec

[ Parent | on HN ]