Ask HN: Who is using small OS LLMs in production?

Re:Ask HN: Who is using small OS LLMs in production? by metiscus on Wednesday August 02, 2023 @09:20AM (#36970504)

LLAMA 2 appears to require no cost because it can be run locally. The license does mention that if you are using facilitating than 700 million users that you have to negotiate for a different license or something like that but for most peoples uses, it would seem that LLAMA 2 is basically "free".

[ Parent | on HN ]

Re:Ask HN: Who is using small OS LLMs in production? by LewisVerstappen on Wednesday August 02, 2023 @09:21AM (#36970533)

Thanks for elaborating. Yeah I was curious how much you'd pay running it on a cloud server in a production-type scenario. Thanks

[ Parent | on HN ]

Re:Ask HN: Who is using small OS LLMs in production? by moffkalast on Wednesday August 02, 2023 @09:33AM (#36970665)

Well it's free as in free hops. You still gotta buy/rent the brewery to make the free beer.

[ Parent | on HN ]

Re:Ask HN: Who is using small OS LLMs in production? by metiscus on Wednesday August 02, 2023 @09:39AM (#36970740)

Exactly. It isn't free as in FSF definitional freedom, you still can't do certain things with it etc but it is unencumbered by external costs unless you exceed a certain usage threshold.

[ Parent | on HN ]

Re:Ask HN: Who is using small OS LLMs in production? by smallerfish on Wednesday August 02, 2023 @09:57AM (#36970972)

Right, but can you scale GPUs for cheaper than OpenAI charges to use their APIs? 3.5 is _cheap_, and perfectly good for many use cases.

[ Parent | on HN ]