So far pretty awesome - simpler first pass, integrated their BitLinear layer into my training setup, and got a run with just the MLPs swapped to see how things are- at comparable parameter counts (3.2B), the BitLinear MLP model is slightly lower loss!