ketothekingdom
Re:I trained a 360M model to pretend to be a terminal
on Saturday October 24, 2026 @10:41PM (#50227766)
nice :)
Re:You could have invented transformers from memory
on Monday October 19, 2026 @05:48PM (#50158531)
I use a combination of Hugo with obsidian-style syntax for everything
Re:You could have invented transformers from memory
on Saturday October 17, 2026 @01:04PM (#50132862)
Definitely! It's more of an intuition but doesn't explain what vector gets put in the weights and the analysis doesn't involve the learning dynamics at all. I link some papers at the end that do think about how memory relates to the dataset. But the architecture having this storing+lookup property probably is important in helping transformers remember so much information.
Re:You could have invented transformers from memory
on Friday October 16, 2026 @11:36PM (#50127675)
Thank you! It's been sitting in my head for so long. I'm glad I finally got to share it with people
Re:You could have invented transformers from memory
on Friday October 16, 2026 @11:35PM (#50127671)
That's such a neat idea! Now I want to try it out too
Re:You could have invented transformers from memory
on Friday October 16, 2026 @08:46PM (#50126675)
Hey author here! I have a short summary here if people find it helpful: https://x.com/heyyalexwang/status/2111158818730397932?s=46
The Model Is the Company
Re:Neural audio codecs: how to get audio into LLMs
on Tuesday October 21, 2025 @09:31PM (#45663963)
Cartesia is doing constant time models for audio: https://cartesia.ai/