News for hackers. Stuff that matters.
Re:Web LLM – WebGPU Powered Inference of Large Language Models by wokwokwok on Saturday April 15, 2023 @08:22PM (#35586006)
Most people are distributing diff files like this https://github.com/lm-sys/FastChat#vicuna-weights

Ie. Here’s a diff. Go find your own copy of the llama model and apply this patch.

…and hoping that’s good enough to get away with; even though you can’t really argue it’s not distributing a derivative work in part.

Distribution of the actual model (eg. Running it in your browser) seems to usually result in “and then Meta slaps you with a takedown notice”.

Re:Web LLM – WebGPU Powered Inference of Large Language Models by Tepix on Sunday April 16, 2023 @02:51PM (#35593111)
I've only seen vicuna do it so far but this is the right way for now.