Re:Fifteen Times Faster Than Llama 2: Introducing DeciLM LLM...
by slimsag on Friday September 15, 2023 @09:40PM
(#37531169)
> permissively licensed, open-source foundation LLM
Sounds great
> This license is, in part, based on the LLAMA 2 Community License Agreement, with a series of modifications. Use of DeciLM for hosted services may require a separate license.
Ah, so a bait-and-switch. It's not open source, and not permissively licensed.
Re:Fifteen Times Faster Than Llama 2: Introducing DeciLM LLM...
by karim79 on Friday September 15, 2023 @10:11PM
(#37531353)
Three pricing tiers and no listed prices on the "pricing" page. Only the option to talk to sales. So, depending on the outcome of the conversation one could end up paying less for the Enterprise tier than for the Basic one.
Re:Fifteen Times Faster Than Llama 2: Introducing DeciLM LLM...
by SubiculumCode on Friday September 15, 2023 @10:14PM
(#37531367)
Well, I admit that I now feel bad for posting it to HN; I should have looked at the actual details of the license rather than the headline they are trying to sell.
Re:Fifteen Times Faster Than Llama 2: Introducing DeciLM LLM...
by brucethemoose2 on Saturday September 16, 2023 @05:20PM
(#37538826)
> how many iterations are usual these days?
I dunno about a precise number...Less than 30? The UniPC scheduler in particular can get great results below 20 iterations. Karras DPM schedulers are pretty good with 30.
> In terms of unmodified diffusers, you mean fine tuned to whatever they are working on?
No, people hack non-default augmentations onto the vanilla diffusers (or SAI) pipe to increase speed or increase quality. Some examples included the channels last memory format, downsampling some stages to FP16, TomeSD, self attention, and some more user-guided augmentations like controlnet or tiled diffusion.
"alternate" runtimes like the ones available via torch.compile or AITemplate provide some pretty massive speed gains (~2x) if you need them, but are less common since they are hard to install and HF diffusers only (where most of the SD community uses the original SAI implementation).
Just as an example implementation, VoltaML is quite speedy out-of-the-box, with most of the sane optimizations already implemented, though its missing some features (like tiled diffusion/area prompting) that the SAI based pipelines have: https://github.com/VoltaML/voltaML-fast-stable-diffusion
Re:Fifteen Times Faster Than Llama 2: Introducing DeciLM LLM...
by yieldcrv on Friday September 15, 2023 @09:57PM
(#37531268)
Is there any community or resource that keeps up with all combinations of performance improvements?
another day, another model. okay how is it quantized to 4bit, what about on AMD chips with ROCm, what about with some other forms of paging I’ve seen, tokens/sec, does it pass AI grade school and higher education, is it fine tuned?
equivalent cost per hour if using various cloud solutions when optimized?
Re:Fifteen Times Faster Than Llama 2: Introducing DeciLM LLM...
by nickpsecurity on Friday September 15, 2023 @10:18PM
(#37531389)
Best one I know for local AI's is below. The people who make the quantizations and OSS software often post there. You can find a lot on models, building rigs, etc using search.
Re:Fifteen Times Faster Than Llama 2: Introducing DeciLM LLM...
by dTal on Saturday September 16, 2023 @10:44AM
(#37535214)
Well I hope the community decides not to press the issue for a good while, lulling companies into a false sense of security that they can release large high-quality models with "licenses" attached, before the seminal court case that rules that language models are not in fact copyrightable and no one can stop us doing anything we like with such data.
Sounds great
> This license is, in part, based on the LLAMA 2 Community License Agreement, with a series of modifications. Use of DeciLM for hosted services may require a separate license.
Ah, so a bait-and-switch. It's not open source, and not permissively licensed.
No single indicator of a starting/base price.
Still, a free for personal use model is better than none. Hopefully, we will soon have some adaptation from Llama 2 that matches it, fairly soon.
I haven’t personally tested either yet.
Neither is unmodified HF diffusers (or worse, the SAI implementation).
Some of the techniques they describe are neat though
I dunno about a precise number...Less than 30? The UniPC scheduler in particular can get great results below 20 iterations. Karras DPM schedulers are pretty good with 30.
> In terms of unmodified diffusers, you mean fine tuned to whatever they are working on?
No, people hack non-default augmentations onto the vanilla diffusers (or SAI) pipe to increase speed or increase quality. Some examples included the channels last memory format, downsampling some stages to FP16, TomeSD, self attention, and some more user-guided augmentations like controlnet or tiled diffusion.
"alternate" runtimes like the ones available via torch.compile or AITemplate provide some pretty massive speed gains (~2x) if you need them, but are less common since they are hard to install and HF diffusers only (where most of the SD community uses the original SAI implementation).
Just as an example implementation, VoltaML is quite speedy out-of-the-box, with most of the sane optimizations already implemented, though its missing some features (like tiled diffusion/area prompting) that the SAI based pipelines have: https://github.com/VoltaML/voltaML-fast-stable-diffusion
another day, another model. okay how is it quantized to 4bit, what about on AMD chips with ROCm, what about with some other forms of paging I’ve seen, tokens/sec, does it pass AI grade school and higher education, is it fine tuned?
equivalent cost per hour if using various cloud solutions when optimized?
https://www.reddit.com/r/localLLaMA/
The license explicitly blocks “Hosting Use” and does not even differentiate between “model as an API” and in-application use.
I hope the community rejects this kind of BS.