What if AI doesn't need more RAM but better math?

Re:What if AI doesn't need more RAM but better math? by fittingopposite on Monday March 30, 2026 @03:18AM (#47571392)

Re continuous fine-tuning: how do you avoid catastrophic forgetting in your proposal?

[ Parent | on HN ]

Re:What if AI doesn't need more RAM but better math? by SphericalCowww on Tuesday March 31, 2026 @05:21AM (#47584727)

My understanding is that this is what the LoRAs are for; my belief is that they serve as "memory" to their live observations (a more NN-like cache, say), while the main LLM remains unchanged. These LoRAs are also weighted, so that LoRAs irrelevant to the current task will not be trained, while the relevant LoRAs will be reinforced.

But I never built it, so I am not sure if such an emergent state will appear or not.

[ Parent | on HN ]

Re:What if AI doesn't need more RAM but better math? by SphericalCowww on Friday April 03, 2026 @07:34AM (#47625489)

I put my ideas here in case you are interested: https://github.com/SphericalCowww/ML_LunaLoRA

[ Parent | on HN ]