The Era of 1-bit LLMs: ternary parameters for cost-effective computing

Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by gryn on Wednesday February 28, 2024 @08:20AM (#39537481)

the reason why digital/numeric processing won is the power loss in the analog world. when design an analog circuit the next processing stage you add at the end has impact on the ones before it.

this then require a higher skill from the engineers/consumers.

if you want to avoid that you need to add op-amps with a gain of 1 at the boundary of each one, this also that care of the power loss at each stage.

the other part is that there's a limit of to the amount of useful information/computation you can do with analog processing too once you take into account voltage noise. when you do a comparison there are stages where analog win but also place where where digital wins.

I'll edit later this with a link to some papers that discuss these topics if I manage to find them in my mess.

[ Parent | on HN ]

Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by dazed_confused on Wednesday February 28, 2024 @12:58PM (#39541487)

Good explanation. When I was working at a semiconductor manufacturer, our thresholds were like 0 - 0.2V to 0.8 - 1.0V. Additionally, if you look at QLC SSDs, their longevity is hugely degraded. Analog computing is non-trivial, to say the least.

[ Parent | on HN ]

Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by im3w1l on Wednesday February 28, 2024 @01:09PM (#39541627)

For the specific case of neural networks they seem to be very resistant to noise. That's why quantization works in the first place.

[ Parent | on HN ]

Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by eru on Wednesday February 28, 2024 @09:11PM (#39545827)

You also have literal power losses, as in waste heat, to deal with.

See https://news.ycombinator.com/item?id=39545817

[ Parent | on HN ]