Re:Emacs-copilot: Large language model code completion for E...
by looofooo on Sunday December 31, 2023 @07:28AM
(#38823609)
Can I run the llm on a ssh server and use it with this plugin?
[ Parent
| on HN ]
Re:Emacs-copilot: Large language model code completion for E...
by jart on Sunday December 31, 2023 @07:38AM
(#38823654)
I don't see why not. You'd probably just change this code:
(with-local-quit
(call-process "wizardcoder-python-34b-v1.0.Q5_K_M.llamafile"
nil (list (current-buffer) nil) t
"--prompt-cache" cash
"--prompt-cache-all"
"--silent-prompt"
"--temp" "0"
"-c" "1024"
"-ngl" "35"
"-r" "```"
"-r" "\n}"
"-f" hist))
To be something like this instead: (with-local-quit
(call-process "ssh" hist (list (current-buffer) nil) t
"hostname"
"wizardcoder-python-34b-v1.0.Q5_K_M.llamafile"
"--prompt-cache" cash
"--prompt-cache-all"
"--silent-prompt"
"--temp" "0"
"-c" "1024"
"-ngl" "35"
"-r" "```"
"-r" "\n}"
"-f" "/dev/stdin"))
I'd also change `cash` to replace '/' with '_' and prefix it with "/tmp/" so remote collective caching just works.