News for hackers. Stuff that matters.
Re:Two new Gemini models, reduced 1.5 Pro pricing, increased... by TIPSIO on Tuesday September 24, 2024 @01:27PM (#41638882)
I do like the trend.

Imagine if Anthropic or someone eventually release a Claude 3.5 but at like a whopping 10x its current speed.

Would be incredibly more useful and game changing than a slow o1 model that may or not be x percent smarter.

Re:Two new Gemini models, reduced 1.5 Pro pricing, increased... by sgt101 on Tuesday September 24, 2024 @01:30PM (#41638929)
We might see that with the inference ASICs later this year I guess?
Re:Two new Gemini models, reduced 1.5 Pro pricing, increased... by xendipity on Tuesday September 24, 2024 @01:43PM (#41639045)
Ooh, what are these ASICs you're talking about? My understanding was that we'll see AMD/Nvidia gpus continue to be pushed and very competitive as well as have new system architectures like cerebras or grok. I haven't heard about new compute platforms framed as ASICs.
Re:Two new Gemini models, reduced 1.5 Pro pricing, increased... by Workaccount2 on Tuesday September 24, 2024 @02:04PM (#41639279)
Cerebras has ridiculously large LLM ASICs that can hit crazy speeds. You can try it with llama 8B and 70B:

https://inference.cerebras.ai/

It's pretty fast, but my understanding is that it is still too expensive even accounting for the speed-up.

Re:Two new Gemini models, reduced 1.5 Pro pricing, increased... by throwup238 on Tuesday September 24, 2024 @02:15PM (#41639397)
Is Cerebras an integrated circuit or more an integrated wafer? :-)

And yeah their cost is ridiculous, on the order for high 6 to low 7 figures per wafer. The rack alone looks several times more expensive than the 8x NVIDIA pods [1]

[1] https://web.archive.org/web/20230812020202/https://www.youtu...

Re:Two new Gemini models, reduced 1.5 Pro pricing, increased... by campers on Wednesday September 25, 2024 @12:50AM (#41643796)
https://www.etched.com/announcing-etched

I think there's another one but I can't remember the name of it.

Also a bit further out is https://spectrum.ieee.org/superconducting-computer

"Instead of the transistor, the basic element in superconducting logic is the Josephson-junction."

Re:Two new Gemini models, reduced 1.5 Pro pricing, increased... by bangaladore on Tuesday September 24, 2024 @02:26PM (#41639511)
Sonnet 3.5 is fast for its quality. But yeah, it's nowhere near Google's flash models. But I assume that is largely just because its a much smaller model.