Re:Ask HN: Is anyone doing anything cool with tiny language ...
by Sohcahtoa82 on Tuesday January 21, 2025 @06:50PM
(#42786773)
Not even on the edge. That's something you could train on a 2 GB GPU.
The general guidance I've used is that to train a model, you need an amount of RAM (or VRAM) equal to 8x the number of parameters, so a 0.125B model would need 1 GB of RAM to train.
The general guidance I've used is that to train a model, you need an amount of RAM (or VRAM) equal to 8x the number of parameters, so a 0.125B model would need 1 GB of RAM to train.