News for hackers. Stuff that matters.
Re:How Is LLaMa.cpp Possible? by pocketarc on Wednesday August 16, 2023 @05:37AM (#37144694)
I'd be more inclined to believe that they're dropping down to gpt-3.5-turbo based on some heuristic, and that's why sometimes it gives you "dumber" responses. If you can serve 5/10 requests with 3.5 by swapping only the "easy" messages out, you've just cut your costs by nearly half (3.5 is like 5% of the cost of 4).
Re:How Is LLaMa.cpp Possible? by vbezhenar on Wednesday August 16, 2023 @06:36AM (#37145099)
Serving me ChatGPT 3.5 when I explicitly requesting ChatGPT 4 sounds like a very bad move? They're not marketing it like "ChatGPT Basic" and "ChatGPT Pro".