Re:Microgpt
by WithinReason on Sunday March 01, 2026 @04:21AM
(#47205095)
I made an explicit reverse pass (no autodiff), it was 8x faster in Python
[ Parent
| on HN ]
Can you share a link?
Re:Microgpt
by WithinReason on Monday March 02, 2026 @11:54AM
(#47220542)
https://www.ideone.com/VAz4Nn
Doesn't run inside IDEone due to the external download link, but you can copy&paste the code over
I made an explicit double-reverse pass (no code!), it was 80x faster in my head!
"I've got an ipod -- In My Mind"
Re:Microgpt
by WithinReason on Monday March 02, 2026 @04:26PM
(#47224335)
code here, it's just not interesting to look at:
tradeoff worth naming: you avoid the autodiff graph overhead (hence the speedup), but any architecture change means rewriting every gradient by hand. fine for a pedagogical project, but that's exactly why autodiff exists.