You could run this analysis on a laptop very easily.
The pile-10k dataset they used for analysis is 33MB, and GPT2 runs ok on a CPU. For the full 10K analysis it's probably quicker to get a GPU though.
Re:We Found an Neuron in GPT-2
by josephmiller on Thursday February 16, 2023 @09:37PM
(#34830004)
Author here!
This is correct. We did almost all this work on a Macbook Pro. Although for the pile-10k dataset analysis we used an A100 GPU because it would take many hours to run on a laptop.
Does this hypothetical laptop have a GPU? StableDiffusion is in this realm of "stuff" and is runnable on consumer GPU systems. It's a bit of trouble to get set Yu p if you're not a python dev (and kinda still is if you are) but it's a pretty neat ML model to play around with.
Re:We Found an Neuron in GPT-2
by RobotToaster on Thursday February 16, 2023 @06:09PM
(#34828014)
My 2 year old gaming laptop runs stablediffusion just fine. Only having a gtx 1660 ti 6gb it's a little at the limit of what will run it though.
For sure, and even training is very doable on consumer hardware these days. Techniques like Dreambooth and LoRA have dramatically lowered the compute cost of finetuning large models on specific concepts. A recent GPU can train Stable Diffusion models on a concept using LoRA in < 30 minutes.
Ie this seems like such a cool area to be in but the data volumes required are huge, complex, etc. Code is simple, cheap, lean, etc by comparison.
Do we have any insight on how this area of research could be usable with less hardware and data? Is there a visible future where a guy and a laptop can make a big program? (without depending on tech getting small/cheap in 50 years or w/e)