News for hackers. Stuff that matters.
Re:You can now train a 70B language model at home by Nouser76 on Friday March 08, 2024 @11:27AM (#39642662)
Is there any framework/system that distributes the work across multiple GPUs on different computers over a network (LAN or WAN)? I'm not concerned much about latency or generation time, but would love to train or load up huge models and send jobs to run overnight.
Re:You can now train a 70B language model at home by jph00 on Friday March 08, 2024 @09:11PM (#39648825)
Yes, you can run FSDP/QLoRA over multi-node. There's a slurm script in the repo showing how to do it.