News for hackers. Stuff that matters.

ayush-garg

Re:Show HN: A lightweight AI gateway to 100+ models, in TS on Monday January 08, 2024 @08:58AM (#38911960)
A few months back, we embarked on what seemed like another regular project. Little did we know, it would turn into such a game changer for LLM deployments.

Feels surreal that this gateway is already processing upwards of 3B tokens a day in such a short time.