News for hackers. Stuff that matters.

A comment on OpenArch – PyTorch implementations of modern LLM architectures.

Re:OpenArch – PyTorch implementations of modern LLM architec... by sarra01 on Monday September 14, 2026 @05:59AM (#49694368)
hey, Building these from scratch in pure PyTorch is honestly the best way to deeply understand the paper details. something better than simply implementing a traditional Transformer or GPT-2，As an individual maintainer, will be able to keep up with future model updates?
[ Parent | on HN ]
Re:OpenArch – PyTorch implementations of modern LLM architec... by anuj0456 on Monday September 14, 2026 @07:41AM (#49695183)
will try my best, but open for contribution.
[ Parent | on HN ]
Re:OpenArch – PyTorch implementations of modern LLM architec... by kccqzy on Monday September 14, 2026 @10:32AM (#49697531)
I think depending on your perspective it might be even better to eschew PyTorch and implement with pure NumPy. Sure it will be slow, but perhaps as a pedagogical exercise it is more worthwhile.
[ Parent | on HN ]
Re:OpenArch – PyTorch implementations of modern LLM architec... by sarra01 on Sunday September 27, 2026 @10:31PM (#49872888)
That makes sense. NumPy does make the mechanism clearer, but aside from learning purposes, it might be a bit slow.
[ Parent | on HN ]