News for hackers. Stuff that matters.
Re:You can now train a 70B language model at home by paul_mk1 on Saturday March 09, 2024 @05:22PM (#39655176)
Goes back before then. This got popularized by BinaryConnect in 2015, and groups were training binary networks as early as 2011.

You are probably referring to XNOR net, and the novel piece there was also using binary activations (which bitnet is not).

So as far as I can tell, bitnet is basically BinaryConnect applied to LLMs.

https://arxiv.org/abs/1511.00363

Re:You can now train a 70B language model at home by gumby on Sunday March 10, 2024 @07:46PM (#39663436)
Thanks for your informative comment. What HN is for!