News for hackers. Stuff that matters.

A comment on Launch HN: Deepsilicon (YC S24) – Software and hardware for ternary transformers.

Re:Launch HN: Deepsilicon (YC S24) – Software and hardware f... by Havoc on Tuesday September 10, 2024 @11:00AM (#41501462)
> This represents an almost 8x compression ratio for every weight matrix in the transformer model

Surely you’d need more ternary weights though to achieve same performance outcome?

A bit like a Q4 quant is smaller than a Q8 but also tangibly worse so the “compression” isn’t really like for like

Either way excited about more tenary progress.

[ Parent | on HN ]
Re:Launch HN: Deepsilicon (YC S24) – Software and hardware f... by areddyyt on Tuesday September 10, 2024 @11:43AM (#41501943)
We do quantization-aware training, so the model should minimize the loss w.r.t. the ternary weights, hence no degradation in performance.
[ Parent | on HN ]