Re:Microsoft BitNet: 100B Param 1-Bit model for local CPUs
by WithinReason on Wednesday March 11, 2026 @12:35PM
(#47337798)
yes but this is not 1 bit matmul, it's 1.58 bits with expensive unpacking
Re:Microsoft BitNet: 100B Param 1-Bit model for local CPUs
by ismailmaj on Wednesday March 11, 2026 @01:02PM
(#47338172)
The title and the repo uses 1-bit when it means 1.58 bits tertiary values, it doesn't change any of my arguments (still xors and pop_counts).
Re:Microsoft BitNet: 100B Param 1-Bit model for local CPUs
by WithinReason on Wednesday March 11, 2026 @01:56PM
(#47338902)
How do you do ternary matmul with popcnt on 1.58 bit packed data?
Re:Microsoft BitNet: 100B Param 1-Bit model for local CPUs
by ismailmaj on Wednesday March 11, 2026 @02:09PM
(#47339074)
Assuming 2 bit per values (first bit is sign and second bit is value).
actv = A[_:1] & B[_:1]
sign = A[_:0] ^ B[_:0]
dot = pop_count(actv & !sign) - pop_count(actv & sign)
It can probably be made more efficient by taking a column-first format.
Since we are in CPU land, we mostly deal with dot products that match the cache size, I don't assume we have a tiled matmul instruction which is unlikely to support this weird 1-bit format.
Re:Microsoft BitNet: 100B Param 1-Bit model for local CPUs
by anematode on Wednesday March 11, 2026 @04:24PM
(#47341016)
Haven't looked closely, but on modern x86 CPUs it might be possible to do much better with the gf2affineqb instructions, which let us do 8x8 bit matrix multiplications efficiently. Not sure how you'd handle the 2-bit part, of course.
Re:Microsoft BitNet: 100B Param 1-Bit model for local CPUs
by WithinReason on Thursday March 12, 2026 @03:48AM
(#47347703)
This is 11 bit ops and a subtract, which I assume is ~11 clocks, while you can just do:
l1 = dot(A[:11000000],B[:11000000]) l2 = dot(A[:00110000],B[:00110000]) l3 = dot(A[:00001100],B[:00001100]) l4 = dot(A[:00000011],B[:00000011])
result = l1 * 64 + l2 * 16 + l3 * 4 + l4
which is 8 bit ops and 4x8 bit dots, which is likely 8 clocks and less serial dependence
typically for 1-bit matmul, you can get away with xors and pop_counts which should have a better throughput profile than FMA when taking into account the SIMD nature of the inputs/outputs.