News for hackers. Stuff that matters.
Re:Google CodeGemma: Open Code Models Based on Gemma [pdf] by tosh on Tuesday April 09, 2024 @09:56AM (#39979479)
It's really sad that Cursor does not support local models yet (afaiu they fetch the URL you provide from their server). Is there a VS Code plugin or other editor that does?

With models like CodeGemma and Command-R+ it makes more and more sense to run them locally.

Re:Google CodeGemma: Open Code Models Based on Gemma [pdf] by ado__dev on Tuesday April 09, 2024 @10:51AM (#39980075)
Cody supports local inference with Ollama for both Chat and Autocomplete. Here's how to set it up: https://sourcegraph.com/blog/local-chat-with-ollama-and-cody :)
Re:Google CodeGemma: Open Code Models Based on Gemma [pdf] by tosh on Tuesday April 09, 2024 @10:56AM (#39980135)
ty for the pointer!
Re:Google CodeGemma: Open Code Models Based on Gemma [pdf] by ericskiff on Tuesday April 09, 2024 @10:18AM (#39979693)
I've been playing with Continue: https://github.com/continuedev/continue
Re:Google CodeGemma: Open Code Models Based on Gemma [pdf] by tosh on Tuesday April 09, 2024 @10:21AM (#39979729)
ty for the pointer!
Re:Google CodeGemma: Open Code Models Based on Gemma [pdf] by tosh on Tuesday April 09, 2024 @10:21AM (#39979726)
Re:Google CodeGemma: Open Code Models Based on Gemma [pdf] by sp332 on Tuesday April 09, 2024 @10:10AM (#39979619)
According to https://forum.cursor.sh/t/support-local-llms/1099/7 the Cursor servers do a lot of work in between your local computer and the model. So porting all that to work on users' laptops is going to take a while.
Re:Google CodeGemma: Open Code Models Based on Gemma [pdf] by knoopx on Wednesday April 10, 2024 @04:18AM (#39988255)
https://github.com/huggingface/llm-vscode

  "llm.backend": "ollama",
  "llm.url": "http://localhost:11434/api/generate",
  "llm.modelId": "codegemma:2b-code-q8_0",
  "llm.configTemplate": "Custom",
  "llm.fillInTheMiddle.enabled": true,
  "llm.fillInTheMiddle.prefix": "<|fim_prefix|>",
  "llm.fillInTheMiddle.middle": "<|fim_middle|>",
  "llm.fillInTheMiddle.suffix": "<|fim_suffix|>",
  "llm.tokensToClear": ["<|fim_prefix|>", "<|fim_middle|>", "<|fim_suffix|>", "<|file_separator|>"],