News for hackers. Stuff that matters.

A comment on The Era of 1-bit LLMs: ternary parameters for cost-effective computing. Parent comment

Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by trebligdivad on Wednesday February 28, 2024 @12:51PM (#39541383)
Trinary however is an interesting middle; people have built trinary hardware long ago; it feels like you could make natively trinary hardware for something like this; it might even be quite a win.
[ Parent | on HN ]
Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by int_19h on Wednesday February 28, 2024 @03:02PM (#39542784)
People haven't built reliable ternary electronics, though. Soviets tried with Setun, but they eventually had to resort to emulating each trit with two hardware bits (and wasting one state out of the possible four).
[ Parent | on HN ]
Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by eru on Wednesday February 28, 2024 @09:11PM (#39545833)
If you are are using two bits anyway, you might as well represent (-2, -1, 0, 1) instead of ternary?
[ Parent | on HN ]
Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by int_19h on Wednesday February 28, 2024 @10:31PM (#39546144)
Sure, but then you lose the symmetry that makes trits so convenient for many things.
[ Parent | on HN ]
Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by thsksbd on Wednesday February 28, 2024 @01:24PM (#39541814)
Can you make a "CMOS" three voltage level circuit though? One where the only current flow is when the state changes?

Im not in this field but that's a question that's been bugging me for a while. Off you can't do this wouldn't energy consumption balloon?

[ Parent | on HN ]
Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by neomantra on Thursday February 29, 2024 @07:41AM (#39548582)
My friend was working on this in the mid-90s at Texas Instruments. Not sure what the underlying semiconductors were, but it did involve making ternary logic via voltage levels. Just searched a bit and found this TI datasheet which might be an example of it (high logic, low logic, high impedance), but maybe not: https://www.ti.com/lit/ds/symlink/sn74act534.pdf
[ Parent | on HN ]