News for hackers. Stuff that matters.
Re:Llama3 implemented from scratch by pshc on Sunday May 19, 2024 @03:39PM (#40409259)
My wild guess is that adjusting the shape before each step is not worth the speed hit. Uniform structures make GPUs go brrrrr
Re:Llama3 implemented from scratch by astrange on Sunday May 19, 2024 @04:52PM (#40409660)
It's also easier to train and in particular easier to parallelize.