I can see some companies not having the technical ability to pull off offline LLMs, so this product could cater to that market.
Re:Introducing ChatGPT Enterprise
by Patrick_Devine on Monday August 28, 2023 @01:46PM
(#37297818)
Maybe, but that's why things like ollama.ai are trying to fill the gap. It's simple, and you don't need all of the heavy weight enterprise crap if nothing ever leaves your system.
Re:Introducing ChatGPT Enterprise
by rangledangle on Monday August 28, 2023 @01:51PM
(#37297897)
Less technical companies throw money at problems to solve them. Like mine, sadly... Even if it takes a small amount of effort, companies will throw money for zero effort.
Zero execution risk, rather than zero effort. There’s always a 10% chance that implementation goes on forever and spending some money eliminates that risk.
why should they solve it? if it's not a core competency, just buy it.
I'm really not sure at all this can be interpreted as them being startled at LLama 2 at all.
From the very beginning everyone knew data privacy & security would be one of the main issues for corporations.
Re:Introducing ChatGPT Enterprise
by willsmith72 on Monday August 28, 2023 @01:56PM
(#37297973)
most teams don't want to self-host, and definitely don't want to have to run on-device eating up their ram
Re:Introducing ChatGPT Enterprise
by lancesells on Monday August 28, 2023 @02:06PM
(#37298145)
I get the self-host part, but if you had a dedicated machine would the ram be an issue? Can you run it on a machine with like 128GB of ram or the GPU equivalent?
Re:Introducing ChatGPT Enterprise
by whimsicalism on Monday August 28, 2023 @02:04PM
(#37298109)
There is no reason these models will be selfhost only.
Re:Introducing ChatGPT Enterprise
by willsmith72 on Monday August 28, 2023 @02:16PM
(#37298305)
agreed, and I can't wait for gpt4 to have great competition in terms of ease, price and performance. I was responding to this
> something that should just be completely on-device or self-hosted if you don't trust cloud-based AI models like ChatGPT Enterprise and want it all private and low cost
Re:Introducing ChatGPT Enterprise
by sebzim4500 on Monday August 28, 2023 @01:59PM
(#37298022)
Llama 2 is nowhere near the capability of GPT-4 for general purpose tasks
Re:Introducing ChatGPT Enterprise
by screamingninja on Monday August 28, 2023 @02:08PM
(#37298175)
> This sound like a huge waste of money for something that should just be completely on-device or self-hosted
I can imagine this argument being made repeatedly over the past several decades whenever anyone makes a decision to use any paid cloud service. There is a value in self-hosting FOSS services and managing it in house and there is a value in letting someone else manage it for you. Ultimately it depends on the business use case and how much effort / risk you are willing to handle.
Re:Introducing ChatGPT Enterprise
by YetAnotherNick on Monday August 28, 2023 @02:09PM
(#37298190)
If you could offer stable 70B llama API at half the price of ChatGPT API I would pay for it. I know HN likes to believe everything is close to $0, but it is hardly the case.
(Not affiliated) https://together.ai/pricing
Re:Introducing ChatGPT Enterprise
by YetAnotherNick on Monday August 28, 2023 @04:38PM
(#37300104)
So it is 50% more expensive than OpenAI. Even if that was comparable it proves my point that you can hardly do it for "cost close to $0".
We offer Llama-2-70B-chat api[0] at Deep Infra. $1 per 1M tokens, support streaming and OpenAI compatible endpoint[1].
0. https://deepinfra.com/meta-llama/Llama-2-70b-chat-hf
1. https://deepinfra.com/docs/advanced/openai_api
This sound like a huge waste of money for something that should just be completely on-device or self-hosted if you don't trust cloud-based AI models like ChatGPT Enterprise and want it all private and low cost.
But either way, Meta seems to be already at the finish line in this race and there is more to AI than the LLM hype.