Ask HN: What's a standard way for apps to request text completion as a service?

Re:Ask HN: What's a standard way for apps to request text co... by lcian on Saturday January 10, 2026 @08:09AM (#46565412)

When I'm writing a script that requires some kind of call to an LLM, I use this: https://github.com/simonw/llm.

This is of course cross-platform and works with both models accessible through an API and local ones.

I'm afraid this might not solve your problem though, as this is not an out of the box solution, it requires the user to either provide their own API key or to install Ollama and wire it up on their own.

[ Parent | on HN ]

Re:Ask HN: What's a standard way for apps to request text co... by kristopolous on Saturday January 10, 2026 @09:55AM (#46566203)

I've been working on a more unixy version of his tool I call llcat. Composable, stateless, agnostic, and generic:

https://github.com/day50-dev/llcat

It might help things get closer..

It's under 2 days old and it's already really fundamentally changing how I do things.

Also for edge running look into the LFM 2.5 class of models: https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct

[ Parent | on HN ]

Re:Ask HN: What's a standard way for apps to request text co... by mirror_neuron on Saturday January 10, 2026 @10:43AM (#46566637)

I love this concept. Looks great, I will definitely check it out.

[ Parent | on HN ]

Re:Ask HN: What's a standard way for apps to request text co... by kristopolous on Saturday January 10, 2026 @07:29PM (#46571470)

Please use it and give me feedback. I'm going to give a lightning talk on it tonight at sfvlug

[ Parent | on HN ]

Re:Ask HN: What's a standard way for apps to request text co... by nvader on Saturday January 10, 2026 @10:11AM (#46566333)

I think this is definitely a step in the right direction, and is exactly the kind of answer I was looking for. Thank you!

`llm` gives my tool a standard bin to call to invoke completions, and configuring and managing it is the user's responsibility.

If more tools started expecting something like this, it could become a defacto standard. Then maybe the OS would begin to provide it.

[ Parent | on HN ]