Re:Open source solution replicates ChatGPT training process
by Taek on Monday February 20, 2023 @10:57AM
(#34869302)
For reference, GPT-NeoX is a 20B parameter model, and it runs on 45 GB of VRAM. On an 80 GB A100 you could probably run a 35B parameter model. Maybe 8 A100 cards to do inference on ChatGPT?
Or 32 3090 cards, which would run you under $40k total.
Or 32 3090 cards, which would run you under $40k total.