News for hackers. Stuff that matters.
Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by mjcohen on Wednesday February 28, 2024 @11:51PM (#39546491)
IIRC, Hamming's book "Digital Filters" (1989) has a section on FFTs with only the sign of the coefficient being used. It performed surprisingly well.
Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by thomasahle on Thursday February 29, 2024 @01:22AM (#39546845)
You mean Fast Hadamard Transform?
Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by mitthrowaway2 on Thursday February 29, 2024 @02:36AM (#39547168)
What is the sign of a complex number? Do you mean the phase?
Re:The Era of 1-bit LLMs: ternary parameters for cost-effect... by nine_k on Thursday February 29, 2024 @09:45AM (#39549955)
AFAICT, both the real and imaginary components are from (-1, 0, +1) only. No single sign, but only 8 directions and the center.