News for hackers. Stuff that matters.

A comment on ChatGPT's API Is So Good and Cheap, It Makes Most Text Generating AI Obsolete. Parent comment

Re:ChatGPT's API Is So Good and Cheap, It Makes Most Text Ge... by MacsHeadroom on Saturday March 11, 2023 @08:12PM (#35115117)
It would cost less than $50 of compute to finetune an existing open source LLM to do the job for another $50 of computer. OpenAI's APIs are incredibly expensive compared to powerful open source LLMs which run on very accessible hardware.

As one data point, LLaMA-13B beats GPT-3 175B in benchmarks, runs on a single 8GB VRAM consumer GPU, and takes only 24GB of VRAM to fine tune. (Though this particular model can't be used for commercial purposes.)

[ Parent | on HN ]
Re:ChatGPT's API Is So Good and Cheap, It Makes Most Text Ge... by manmal on Sunday March 12, 2023 @04:25AM (#35117979)
I actually read about LLaMA after posting the above comment, and I do agree with you. You probably can't use it OOTB right now because LLaMA prompts seem harder to get right, but seeing the rate of progress, that will probably be solved in a few weeks.
[ Parent | on HN ]