Fifteen Times Faster Than Llama 2: Introducing DeciLM LLM with Variable GQA

Re:Fifteen Times Faster Than Llama 2: Introducing DeciLM LLM... by yieldcrv on Friday September 15, 2023 @09:57PM (#37531268)

Is there any community or resource that keeps up with all combinations of performance improvements?

another day, another model. okay how is it quantized to 4bit, what about on AMD chips with ROCm, what about with some other forms of paging I’ve seen, tokens/sec, does it pass AI grade school and higher education, is it fine tuned?

equivalent cost per hour if using various cloud solutions when optimized?

[ Parent | on HN ]

Re:Fifteen Times Faster Than Llama 2: Introducing DeciLM LLM... by nickpsecurity on Friday September 15, 2023 @10:18PM (#37531389)

Best one I know for local AI's is below. The people who make the quantizations and OSS software often post there. You can find a lot on models, building rigs, etc using search.

https://www.reddit.com/r/localLLaMA/

[ Parent | on HN ]