News for hackers. Stuff that matters.
Re:g1: Using Llama-3.1 70B on Groq to create o1-like reasoni... by cubefox on Monday September 16, 2024 @07:24AM (#41554927)
It's interesting that DeepMind still publishes this stuff. OpenAI doesn't publish anything of that sort anymore. DeepMind is more research/publication focused, but this is a disadvantage in a competitive landscape where OpenAI and Anthropic can just apply the results of your paper without giving anything back to the research community.
Re:g1: Using Llama-3.1 70B on Groq to create o1-like reasoni... by cabidaher on Monday September 16, 2024 @08:20AM (#41555303)
Anthropic publishes quite a lot too though.
Re:g1: Using Llama-3.1 70B on Groq to create o1-like reasoni... by cubefox on Monday September 16, 2024 @08:40AM (#41555475)
On safety, but no longer on capabilities.
Re:g1: Using Llama-3.1 70B on Groq to create o1-like reasoni... by marricks on Monday September 16, 2024 @08:36AM (#41555440)
> but this is a disadvantage in a competitive landscape

Or it's a unique advantage because this stuff doesn't happen without good researches who may want:

1) Their name in scientific papers

2) They might actually care about the openess of AI

Re:g1: Using Llama-3.1 70B on Groq to create o1-like reasoni... by cubefox on Monday September 16, 2024 @08:38AM (#41555462)
So far it seems to be a disadvantage as DeepMind has fallen behind OpenAI, despite their size, and to some extent even behind Anthropic.
Re:g1: Using Llama-3.1 70B on Groq to create o1-like reasoni... by marricks on Monday September 16, 2024 @08:47AM (#41555540)
They feel behind because they didn't have the smart guy with a new idea a few years back, and HE decided to work at a place which started as open.

Playing catch up and trying to attract talent from the hot-new-thing OpenAI requires incentives beyond lots of money. I contend actually being open helps.

I'm sure that's one reason Facebook has an open source model, scientists can care about ethics and could be attracted to openness.

Re:g1: Using Llama-3.1 70B on Groq to create o1-like reasoni... by michaelt on Monday September 16, 2024 @10:02AM (#41556184)
> They feel behind because they didn't have the smart guy with a new idea a few years back, and HE decided to work at a place which started as open.

The "Attention Is All You Need" guys all worked at Google. Google is where they are despite having the smart guys with a new idea a few years back.

Of course, IMHO it wouldn't have have helped Google if they'd kept the transformer architecture secret. They'd have fumbled it because they didn't realise what they had.

Re:g1: Using Llama-3.1 70B on Groq to create o1-like reasoni... by zozbot234 on Monday September 16, 2024 @11:18AM (#41556951)
Didn't Google have the LaMDA model pretty early, which was even described as "sentient" at some point? That doesn't look "fumbled" to me.
Re:g1: Using Llama-3.1 70B on Groq to create o1-like reasoni... by michaelt on Monday September 16, 2024 @02:25PM (#41558974)
What Google did was sit on their ass, not deigning to release anything. In the meantime, OpenAI became a $150 billion company. And Anthropic came out with Claude, and Facebook with Llama, and Mistral with their models.

Only then did Google realise there might be something to this LLM stuff - so they responded with Bard, a product so poorly received they later had to completely rebrand it. Looks like they didn't have a "sentient" model up their sleeve after all. Then the updated, rebranded model had a bunch of image generation embarrassments of its own.

Admittedly, they have recovered somewhat since then; they're second on some performance leaderboards, which is respectable.

But there was a real tortoise-and-hare situation where they thought they were so far ahead they had time for a nap, until they got overtaken. Any lead they had from inventing transformers and being the only people with TPUs has been squandered.

Re:g1: Using Llama-3.1 70B on Groq to create o1-like reasoni... by cubefox on Monday September 16, 2024 @09:26PM (#41563057)
I have the impression they regarded generative AI as too dangerous. Before the success of ChatGPT, they never considered making PaLM or LaMDA or Chinchilla or Imagen publicly available until they saw themselves in a competitive disadvantage.