bddppq
Re:Llama 3.1 Official Launch
on Tuesday July 23, 2024 @08:48PM (#41052589)
yep with <= 8bit (int8/fp8) quantization
Re:Show HN: Running LLMs in one line of Python without Docker
on Wednesday October 04, 2023 @05:59PM (#37772384)
lepton is at a different layer comparing to llama.cpp, in fact for LLM model files that are of GGUF format, it's using llama.cpp (ctransformers to be precise) as the execution engine
Re:Show HN: Running LLMs in one line of Python without Docker
on Wednesday October 04, 2023 @04:53PM (#37771618)
Are there some examples(prompts) that Falcon 180B is performing better than Llama 70B?