Re:Ask HN: Do people want to AB test LLMs?
by hypoxia87 on Thursday October 10, 2024 @04:53PM
(#41803480)
We've actually found the opposite -- Every client project has been based on GPT4 or Gemini, with one exception for a highly sensitive use case based on Llama3.1.
The main reason is that the APIs represent an excellent cost / performance / complexity tradeoff.
Every project has relied primarily on the big models because the small models just aren't as capable in a business context.
We have found that Gpt4o is very fast, when that's necessary (often it's not), and it's also very cheap (gpt4o batch is ~96% cheaper than the original GPT4). And where cost is a concern and reasoning doesn't need to be as good as possible, gpt4o mini has been excellent too.
The main reason is that the APIs represent an excellent cost / performance / complexity tradeoff.
Every project has relied primarily on the big models because the small models just aren't as capable in a business context.
We have found that Gpt4o is very fast, when that's necessary (often it's not), and it's also very cheap (gpt4o batch is ~96% cheaper than the original GPT4). And where cost is a concern and reasoning doesn't need to be as good as possible, gpt4o mini has been excellent too.