News for hackers. Stuff that matters.

A comment on Llama.cpp 30B runs with only 6GB of RAM now. Parent comment

Re:Llama.cpp 30B runs with only 6GB of RAM now by pharmakom on Saturday April 01, 2023 @07:01AM (#35399210)
I was talking about running in the Web browser. it's not everything, but it's an important part of everything in my book.
[ Parent | on HN ]
Re:Llama.cpp 30B runs with only 6GB of RAM now by sireat on Saturday April 01, 2023 @01:43PM (#35402261)
https://github.com/pyodide/pyodide is pretty amazing for running Python client side in the browser.

You could run notebooks entirely client side https://jupyterlite.readthedocs.io/en/latest/

The startup is slow but otherwise it is pretty functional.

[ Parent | on HN ]