News for hackers. Stuff that matters.
Re:Mistral AI launches Mixtral-Next by bloopernova on Saturday February 17, 2024 @10:36AM (#39410391)
Slightly related question: what's a good coding LLM to run on a 4070 12GB card?

Also, do coding LLMs use treesitter to "understand" code?

Re:Mistral AI launches Mixtral-Next by wiradikusuma on Saturday February 17, 2024 @12:22PM (#39411358)
Another one is https://github.com/WisdomShell/codeshell/blob/main/README_EN... and it has its own IntelliJ plugin.
Re:Mistral AI launches Mixtral-Next by ianschmitz on Saturday February 17, 2024 @10:59AM (#39410613)
I’m pretty new to running these locally, but here’s my understanding:

Best models currently: codellama or deepseek coder. 6.7B or 1B depending on how much latency you can tolerate

Treesittee: from looking at the logs of the chat completions requests for Continue or Twinny extensions for VS Code, they both appear to just send a chunk of the document as well as a special placeholder to indicate where the cursor currently is.

Re:Mistral AI launches Mixtral-Next by sob727 on Saturday February 17, 2024 @10:39AM (#39410424)
I'm also interested in the answer to that.
Re:Mistral AI launches Mixtral-Next by explorigin on Saturday February 17, 2024 @11:01AM (#39410634)
Depends on what you want to use it for. I use deepseek-coder v1 (1.5 is too verbose). I use it like a customized web search to quickly build one-off scripts in python.

If you're wanting something to be your hands so you don't have to type, open-source LLMs and IDE integration is not reliably there yet. Follow the AIDER discord to stay up on the latest in this area.

Re:Mistral AI launches Mixtral-Next by Havoc on Saturday February 17, 2024 @01:23PM (#39411938)
Try deepseek 6.7B
Re:Mistral AI launches Mixtral-Next by fancy_pantser on Saturday February 17, 2024 @01:52PM (#39412247)
> do coding LLMs use treesitter...?

It's up to the app to put that into the context. Generally, coding LLMs do well if you provide them the source tree, graph, search results, notable files, etc in the context. The is how Sourcegraph's Cody product works, for example.