Re:Running large language models like ChatGPT on a single GPU
by nl on Monday February 20, 2023 @07:55PM
(#34875069)
This is exactly right.
The only exception is running things on mobile. There is demand for porting models to run natively on mobile, and somewhat reasonable support for doing this.
The only exception is running things on mobile. There is demand for porting models to run natively on mobile, and somewhat reasonable support for doing this.