Ask HN: Who is using small OS LLMs in production?

Re:Ask HN: Who is using small OS LLMs in production? by danieldk on Wednesday August 02, 2023 @09:40AM (#36970745)

The long-term price difference can be hard to estimate. Suppose that your application is heavily dependent on GPT-4. OpenAI can double your API prices the next term. Or they can decide to stop supporting the model that you wrote all your custom prompts for. Or they could decide to disable your account because they feel like it.

Unless it's for some fringe feature, building your business on OpenAI is probably a considerable (financial) risk in the future.

[ Parent | on HN ]

Re:Ask HN: Who is using small OS LLMs in production? by fauigerzigerk on Wednesday August 02, 2023 @10:27AM (#36971372)

These are definitely very significant risks. But some of those risks are hard to avoid unless you train your own model, which can be prohibitively expensive.

Say you're building on top of llama and Facebook decides not to update it any more or change the licensing terms (again). Say you're building on some other "open source" model and that project dies.

At least you can keep running the existing model rather than getting locked out over night. That's definitely much better. At least it's survivable. But you would still have to find alternatives and review/scrap all your custom prompts.

[ Parent | on HN ]