News for hackers. Stuff that matters.

A comment on Many options for running Mistral models in your terminal using LLM. Parent comment

Re:Many options for running Mistral models in your terminal ... by bugglebeetle on Monday December 18, 2023 @10:09PM (#38691323)
Yeah, I was surprised Ollama was not mentioned as it’s by far the easiest to get started with. If it only had real grammar support, I’d never have to use another library again (it has a JSON mode that generally works, however).
[ Parent | on HN ]
Re:Many options for running Mistral models in your terminal ... by Tomte on Tuesday December 19, 2023 @03:05AM (#38693068)
What is grammar support? I've seen that mentioned several times now. Does it allow to restrict the output to a given template, or am I totally wrong there?
[ Parent | on HN ]
Re:Many options for running Mistral models in your terminal ... by biwills on Tuesday December 19, 2023 @03:34AM (#38693227)
Yes. Here's a quick rundown of grammar in llama.cpp (link on the docs of faraday.dev which runs llama.cpp under the hood)

https://docs.faraday.dev/character-creation/grammars

[ Parent | on HN ]