The problem with tools like this is that somewhere in the chain between you and the LLM are token reducing “features”. Whether it’s the system prompt, a cheaper LLM middleman, or some other cost saving measure.
You’ll never know what that something is. For me, I can’t help but think that I’m getting an inferior service.
You can self host something like https://big-agi.com/ and grab your own keys from various providers. You end up with the above, but without the pitfalls you mentioned.
Re:Gemini 2.5 Flash
by mark_l_watson on Friday April 18, 2025 @09:58AM
(#43728154)
BIG-AI does look cool, and supports a different use case. ABACUS.AI takes your $10/month and gives you credits that go towards their costs of using OpenAI, Anthropic, Gemini, etc. Use of smaller open models use very few credits.
The also support an application development framework that looks interesting but I have never used it.
Re:Gemini 2.5 Flash
by mark_l_watson on Friday April 18, 2025 @09:54AM
(#43728124)
You might be correct about cost savings techniques in their processing pipeline. But they also add functionality: they bake web search into all models which is convenient. I have no affiliation with ABACUS.AI, I am just a happy customer. They currently let me play with 25 models.
Re:Gemini 2.5 Flash
by freedomben on Friday April 18, 2025 @10:29AM
(#43728415)
If anyone from Kagi is on, I'd love to know, does Kagi do that?
I agree that giving away access to expensive models long term is not a good idea on several fronts. Personally, I subscribe to Gemini Advanced and I pay for using the Gemini APIs.
EDIT: a very good deal, at $10/month is https://apps.abacus.ai/chatllm/ that gives you access to almost all commercial models as well as the best open weight models. I have never come close at all to using my monthly credits with them. If you like to experiment with many models the service is a lot of fun.