News for hackers. Stuff that matters.
Re:Gemini: Google's most capable AI model yet by behnamoh on Wednesday December 06, 2023 @11:26AM (#38545998)
I hate this "tierification" of products into categories: normal, pro, max, ultra

Apple does this and it's obvious that they do it to use the "decoy effect" when customers want to shop. Why purchase a measly regular iPhone when you can spend a little more and get the Pro version?

But when it comes to AI, this tierification only leads to disappointment—everyone expects the best models from the FAANGO (including OpenAI), no one expects Google or OpenAI to offer shitty models that underperform their flagships when you can literally run Llama 2 and Mistral models that you can actually own.

Re:Gemini: Google's most capable AI model yet by crazygringo on Wednesday December 06, 2023 @12:01PM (#38546480)
I don't understand -- these are all literally tied directly to performance.

They're tiers of computing power and memory. More performance costs more money to produce. The "nano" can fit on a phone, while the others can't.

Are you really objecting to the existence of different price/performance tiers...? Do you object to McDonald's selling 3 sizes of soft drink? There's nothing "decoy" about any of this.

Re:Gemini: Google's most capable AI model yet by rafram on Wednesday December 06, 2023 @12:42PM (#38547107)
> Do you object to McDonald's selling 3 sizes of soft drink?

Yes, actually, for different reasons - McDonald’s charges only a tiny bit more for the largest size of drink than they do for the smallest (which is easy because soft drinks are a few cents’ worth of syrup and water, and the rest is profit). That pushes people toward huge drinks, which means more sugar, more caffeine, and more addiction.

Re:Gemini: Google's most capable AI model yet by crazygringo on Wednesday December 06, 2023 @12:54PM (#38547258)
But you're not objecting to selling 3 sizes. You're just objecting that the prices aren't far enough apart...
Re:Gemini: Google's most capable AI model yet by mensetmanusman on Wednesday December 06, 2023 @11:43AM (#38546234)
It has to be this way when current LLMs have orders of magnitude electricity cost differences depending on the output you desire.
Re:Gemini: Google's most capable AI model yet by chongli on Wednesday December 06, 2023 @11:36AM (#38546154)
No, it’s not just to use the “decoy effect.” They do this to share development costs across a whole product line. Low volume, expensive products are subsidized by high volume, mass market devices. Without these tiers, they’d be unable to differentiate the products and so lose the margins of the high end products (and their entire reason for existing).

Unless you expect Apple to just sell the high end devices at a loss? Or do you want the high end chips to be sold in the mass market devices and for Apple to just eat the R&D costs?

Re:Gemini: Google's most capable AI model yet by jorvi on Wednesday December 06, 2023 @11:58AM (#38546435)
> They do this to share development costs across a whole product line. Low volume, expensive products are subsidized by high volume, mass market devices

Usually it’s the other way around. Mass market products have thin margins and are subsidized by high end / B2B products because the customers for those products have infinitely deep pockets.

> Or do you want the high end chips to be sold in the mass market devices and for Apple to just eat the R&D costs?

Literally what Steve Jobs was steadfast in :). One iPhone for everyone. He even insisted on the Plus models carrying no extra features.

Re:Gemini: Google's most capable AI model yet by chongli on Wednesday December 06, 2023 @12:08PM (#38546601)
Usually it’s the other way around. Mass market products have thin margins and are subsidized by high end / B2B products because the customers for those products have infinitely deep pockets.

That's usually what I've seen, but the M1 MacBook Air came out first and the M1 Pro and Max came out much later.

Re:Gemini: Google's most capable AI model yet by pixl97 on Wednesday December 06, 2023 @02:36PM (#38548449)
That's commonly caused by things like low yields for the highest end devices/binning not allowing them to make the numbers of the high end products they need.
Re:Gemini: Google's most capable AI model yet by giovannibonetti on Wednesday December 06, 2023 @11:42AM (#38546223)
I think the expensive ones are used when the customer is the user — e.g. ChatGPT Plus (personal) subscription — and the cheap ones when they are not — e.g. customer support service bots.
Re:Gemini: Google's most capable AI model yet by jchw on Wednesday December 06, 2023 @11:32AM (#38546091)
I'm honestly 100% okay with it as long as it's reasonable and not confusing to customers. (Not saying Apple isn't somewhat; I mean, buying a non-Pro iPhone 15 and not being able to view WebM files feels literally fucking insane, and that's apparently how that works, but that's a rant for a different thread.) In cases like this, presumably the idea isn't actually feature-gating, it's scaling up. AI inference costs compute time, and although I have no idea if the inference occurs on special hardware or not, if it does, I can only presume that scaling up the special hardware to meet demand is challenging and very much not like scaling up e.g. a typical web service.

IMO, Tiers can be useful when they make sense and aren't just for artificial market segmentation.

Re:Gemini: Google's most capable AI model yet by dankle on Wednesday December 06, 2023 @11:33AM (#38546116)
I think it depends. It's always worth having a small fast model for some tasks and being able to run it completely offline on a mobile cpu. Maybe not as a chat companion, for for text understanding or indexing all your messages and photos for search, it may be enough.
Re:Gemini: Google's most capable AI model yet by pphysch on Wednesday December 06, 2023 @11:32AM (#38546089)
This isn't "tierificaton" or even premiumization. That may come later.

Large AI models have tight resources requirements. You physically can't use X billion parameters without ~X billion ~bytes of memory.

It makes complete sense to have these 3 "tiers". You have a max capability option, a price-performance scaling option, and an edge compute option.

Re:Gemini: Google's most capable AI model yet by dragonwriter on Wednesday December 06, 2023 @11:58AM (#38546441)
> Large AI models have tight resources requirements. You physically can't use X billion parameters without ~X billion ~bytes of memory.

Well, X billion bits times the parameter bit size. For base models, those are generally 32-bit (so 4X bytes), though smaller quantizations ate possible and widely used for public models, and I would assume as a cost measure for closed hosted models as well.

Re:Gemini: Google's most capable AI model yet by pphysch on Wednesday December 06, 2023 @12:15PM (#38546713)
Hence ~
Re:Gemini: Google's most capable AI model yet by city_guy_1 on Wednesday December 06, 2023 @11:36AM (#38546155)
More expensive things cost more money, not a surprise imo
Re:Gemini: Google's most capable AI model yet by theonlybutlet on Wednesday December 06, 2023 @12:46PM (#38547156)
It's safe to assume there's good reason in this case. Nano runs locally on smartphones. Pro and Ultra will likely be cost and speed.
Re:Gemini: Google's most capable AI model yet by golol on Wednesday December 06, 2023 @12:56PM (#38547287)
Tierification of AI models is not some business strategy, it is a necessary consequence of the reality that AI is massively compute constrained right now. The size of a model is extremely important for inference time and cost. It just doesn't make sense to release one single model when your method will always yield a family of models with increasing size. The customer can choose a model corresponding to their needs.
Re:Gemini: Google's most capable AI model yet by nkohari on Wednesday December 06, 2023 @01:29PM (#38547727)
My guess is they're branding it in this way to obfuscate the number of parameters used, which makes sense because more parameters doesn't necessarily mean a better model. It's kind of like the "number of bits" competition in video game consoles back in the 90s.