News for hackers. Stuff that matters.

A comment on Gorilla-CLI: LLMs for CLI including K8s/AWS/GCP/Azure/sed and 1500 APIs. Parent comment

Re:Gorilla-CLI: LLMs for CLI including K8s/AWS/GCP/Azure/sed... by shishirpatil on Thursday June 29, 2023 @02:48PM (#36524962)
Yes indeed. The models are too computationally expensive to run locally (7.5Billion parameters). Though you could in-principle swap in any local model.
[ Parent | on HN ]
Re:Gorilla-CLI: LLMs for CLI including K8s/AWS/GCP/Azure/sed... by kajecounterhack on Thursday June 29, 2023 @03:00PM (#36525165)
Do y'all have plans to release the model for those who have 16gb graphics cards? (I'm assuming the model is fp16?)
[ Parent | on HN ]
Re:Gorilla-CLI: LLMs for CLI including K8s/AWS/GCP/Azure/sed... by Tostino on Saturday July 01, 2023 @01:43AM (#36547019)
What are you talking about? 7b parameter models run insanely fast if you can offload to gpu, and are entirely reasonable speed if CPU only.
[ Parent | on HN ]