News for hackers. Stuff that matters.
Re:SmolGPT: A minimal PyTorch implementation for training a ... by _joel on Thursday January 30, 2025 @05:55AM (#42876663)
I've seen 3 epochs on some of the finetuning R1 blog posts. It's not my field so not sure how valid that is.
Re:SmolGPT: A minimal PyTorch implementation for training a ... by sva_ on Saturday February 01, 2025 @05:13AM (#42897327)
Yeah, fine-tuning is different from pretraining