The Era of 1-bit LLMs: ternary parameters for cost-effective computing

Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by anon373839 on Wednesday February 28, 2024 @07:41AM (#39537099)

It also means the largest models can be scaled up significantly with the same inference budget.

[ Parent | on HN ]

Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by llm_trw on Wednesday February 28, 2024 @07:49AM (#39537169)

Depends. The only paper they cite for training: https://arxiv.org/pdf/2310.11453.pdf doesn't improve training costs much and most models are already training constrained. Not everyone has $200m to throw at training another model from scratch.

[ Parent | on HN ]

Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by arunk47 on Wednesday February 28, 2024 @04:43PM (#39543826)

Is there any scope for indie builders?

[ Parent | on HN ]

Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by llm_trw on Wednesday February 28, 2024 @07:28PM (#39545247)

Not really. These are slightly better for memory during pre-training and fine turning but not enough to make a 4090 usable even for a 7b model.

[ Parent | on HN ]