News for hackers. Stuff that matters.

A comment on NanoGPT. Parent comment

Re:NanoGPT by wongarsu on Wednesday January 11, 2023 @08:17AM (#34338042)
GPT2 can be run locally (on a somewhat beefy consumer GPU)
[ Parent | on HN ]
Re:NanoGPT by karmajuney on Wednesday January 11, 2023 @12:37PM (#34341434)
Can you add some info on what consumer GPU would be needed for this? Would a 3080 be able to handle this?
[ Parent | on HN ]
Re:NanoGPT by wongarsu on Wednesday January 11, 2023 @04:00PM (#34344727)
Assuming you get the 12GB version of the 3080. A 2080TI is another option. Though you can reduce precision or use one of the smaller GPT2 versions to run on smaller cards as well.
[ Parent | on HN ]
Re:NanoGPT by int_19h on Wednesday January 11, 2023 @06:28PM (#34346597)
Let me slightly rephrase the question: what is the best model that one can run on high-end consumer grade hardware? Let's say RTX 3090.
[ Parent | on HN ]
Re:NanoGPT by minimaxir on Wednesday January 11, 2023 @12:51PM (#34341704)
The original GPT-2 small (the 124M one) can run on a CPU, just slowly and not scalably.
[ Parent | on HN ]