News for hackers. Stuff that matters.
Re:Show HN: Fructose, LLM calls as strongly typed functions by jsight on Wednesday March 06, 2024 @01:42PM (#39619444)
I love the concept, but I'd really prefer being able to use it against local llms (localai, ollama, etc).
Re:Show HN: Fructose, LLM calls as strongly typed functions by edunteman on Wednesday March 06, 2024 @01:45PM (#39619510)
Seems like a great feature (and honestly allows us to do smarter things for strictly structured generation). I'm curious, what's your main motivation for local llms vs hosted APIs?
Re:Show HN: Fructose, LLM calls as strongly typed functions by ramses0 on Wednesday March 06, 2024 @02:24PM (#39620107)
Not wanting to get dinged for $20/mo? Ability to use offline for local home automation (eg: "given a verbal input request, determine the home devices in scope and their on/off state" => "given the current devices state and the verbal request, generate a list of home assistant actions to perform"), using a custom model for the above, etc.
Re:Show HN: Fructose, LLM calls as strongly typed functions by yunohn on Wednesday March 06, 2024 @06:03PM (#39622891)
> Not wanting to get dinged for $20/mo?

OpenAI (& peer) API pricing is honestly quite cheap - I spend way less than 20$ a month for my needs. On the other hand, ChatGPT offers a different experience, so I pay for that too.

https://help.openai.com/en/articles/7127956-how-much-does-gp...

Re:Show HN: Fructose, LLM calls as strongly typed functions by imtringued on Wednesday March 06, 2024 @02:19PM (#39620035)
I'm pretty sure the primary reason is that you don't want an update of the hosted LLM to suddenly break your application without warning.
Re:Show HN: Fructose, LLM calls as strongly typed functions by nate_nowack on Wednesday March 06, 2024 @02:36PM (#39620277)
as with marvin, you can just swap the base url and use any of the oss proxy libs that clone the openai api (but since they don't do function calling [except for mistral i think], its not as good afaik)