News for hackers. Stuff that matters.
Re:QwQ: Alibaba's O1 Like Reasoning LLM by whatever1 on Wednesday November 27, 2024 @05:41PM (#42260531)
Seems that given enough compute everyone can build a near-SOTA LLM. So what is this craze about securing AI dominance?
Re:QwQ: Alibaba's O1 Like Reasoning LLM by littlestymaar on Wednesday November 27, 2024 @05:48PM (#42260583)
> everyone

Let's not disrespect the team working on Qwen, these folks have shown that they are able to ship models that are better than everybody else's in the open weight category.

But fundamentally yes, OpenAI has no other moat than the ChatGPT trademark at this point.

Re:QwQ: Alibaba's O1 Like Reasoning LLM by miohtama on Wednesday November 27, 2024 @05:57PM (#42260649)
They have the moat of being able to raise large funding rounds than everybody else: Access to capital.
Re:QwQ: Alibaba's O1 Like Reasoning LLM by tempusalaria on Wednesday November 27, 2024 @06:33PM (#42260882)
many of these labs have more funding in theory than OpenAI. FAIR, GDM, Qwen all are subsidiaries of companies with $10s of billions in annual profits.
Re:QwQ: Alibaba's O1 Like Reasoning LLM by littlestymaar on Wednesday November 27, 2024 @06:28PM (#42260852)
But access to capital is highly dependent on how interesting you look to investors.

If you don't manage to create a technological gap when you are better funded than your competitors then your attractivity will start being questioned. They have dilapidated their “best team” asset with internal drama, and now that they see their technological advance being demolished by competitors, I'm not too convinced in their prospect for a new funding round unless they show that they can make money out of the consumer market which is where their branding is an unmatched asset (in which case it's not even clear that investing in being the state of the art model is a good business decision).

Re:QwQ: Alibaba's O1 Like Reasoning LLM by seccode on Wednesday November 27, 2024 @08:29PM (#42261518)
Maybe truth here, but also Microsoft didn't lead their latest round, which isn't a great sign for their moat
Re:QwQ: Alibaba's O1 Like Reasoning LLM by lmm on Wednesday November 27, 2024 @10:27PM (#42262155)
Do they have more access to capital than the CCP, if the latter decided to put its efforts behind Alibaba on this? Genuine question.
Re:QwQ: Alibaba's O1 Like Reasoning LLM by nxobject on Wednesday November 27, 2024 @06:13PM (#42260746)
And perhaps exclusive archival content deals from publishers – but that probably works only in an American context.
Re:QwQ: Alibaba's O1 Like Reasoning LLM by _1 on Wednesday November 27, 2024 @05:49PM (#42260596)
It just shows that they're unimaginative and good at copying.
Re:QwQ: Alibaba's O1 Like Reasoning LLM by amazingamazing on Wednesday November 27, 2024 @05:52PM (#42260612)
What’s wrong with copying?
Re:QwQ: Alibaba's O1 Like Reasoning LLM by ralusek on Wednesday November 27, 2024 @07:15PM (#42261147)
If they can only copy, which I'm not saying is the case, then their progress would be bounded by whatever the leader in the field is producing.

In much the same way with an LLM, if it can only copy from its training data, then it's bounded by the output of humans themselves.

Re:QwQ: Alibaba's O1 Like Reasoning LLM by miki123211 on Wednesday November 27, 2024 @10:32PM (#42262177)
> But fundamentally yes, OpenAI has no other moat than the ChatGPT trademark at this point.

That's like saying that CocaCola has no other moat than the CocaCola trademark.

That's an extremely powerful moat to have indeed.

Re:QwQ: Alibaba's O1 Like Reasoning LLM by littlestymaar on Thursday November 28, 2024 @06:17AM (#42264400)
There's a big difference though Coca Cola makes its money from customers out its brands, OpenAI doesn't and it's not clear at all that there is monetization potential in that direction.

Their business case was about being the provider of artificial intelligence to other businesses, not to monetize ChatGPT. There my be an opportunity for a pivot, that would include getting rid of the goal of having the most performant model, cutting training cost to the minimum, and be profitable from there, but I'm not sure it would be enough to justify their $157 Billion valuation.

Re:QwQ: Alibaba's O1 Like Reasoning LLM by anon373839 on Thursday November 28, 2024 @06:28AM (#42264453)
Actually, they don’t have the trademark (yet). USPTO rejected the application:

> [Trademark] Registration is refused because the applied-for mark merely describes a feature, function, or characteristic of applicant’s goods and services.

https://tsdr.uspto.gov/documentviewer?caseId=sn97733261&docI...

Re:QwQ: Alibaba's O1 Like Reasoning LLM by sundarurfriend on Friday November 29, 2024 @02:20AM (#42271661)
It's not nothing, but it's very different from the CocaCola case. Coke is built on getting people used to the specific taste, especially when they're young, and taste is an almost subconscious/System1 thing where people seek comfort in the familiar. ChatGPT is more of a System2 thing, people don't really care whether the answer they seek comes from ChatGPT or from AskJeeves, they just want answers of reasonably similar quality. There's still value in branding and name recognition, but the switching costs are much lower here.
Re:QwQ: Alibaba's O1 Like Reasoning LLM by littlestymaar on Friday November 29, 2024 @03:38AM (#42272156)
People can't tell coke apart from Pepsi in blind tests though, the perceived difference when people are getting asked “Is Pepsi ok?” is 100% conditioned by the attachment to the brand.
Re:QwQ: Alibaba's O1 Like Reasoning LLM by deadbabe on Wednesday November 27, 2024 @05:58PM (#42260656)
AI dominance is secured through legal and regulatory means, not technical methods.

So for instance, a basic strategy is to rapidly develop AI and then say “Oh wow AI is very dangerous we need to regulate companies and define laws around scraping data” and then make it very difficult for new players to enter the market. When a moat can’t be created, you resort to ladder kicking.

Re:QwQ: Alibaba's O1 Like Reasoning LLM by Onavo on Wednesday November 27, 2024 @06:07PM (#42260713)
I believe in china they have been trying to make all data training data

https://www.forbes.com/councils/forbestechcouncil/2024/04/18...

Re:QwQ: Alibaba's O1 Like Reasoning LLM by yazzku on Wednesday November 27, 2024 @10:23PM (#42262135)
Unlike in the US?
Re:QwQ: Alibaba's O1 Like Reasoning LLM by greenavocado on Wednesday November 27, 2024 @10:35PM (#42262192)
Re:QwQ: Alibaba's O1 Like Reasoning LLM by nextworddev on Wednesday November 27, 2024 @08:52PM (#42261650)
1) spreading AI dominance FUD is a good way to get government subsidies

2) not exactly everyone with compute can make LLMs, they need data. Conveniently, the U.S. has been supplying infinite tokens to China through Tiktok.

Re:QwQ: Alibaba's O1 Like Reasoning LLM by nemothekid on Wednesday November 27, 2024 @08:58PM (#42261680)
>Conveniently, the U.S. has been supplying infinite tokens to China through Tiktok

How is this not FUD? What competitive advantage is China seeing in LLM training through dancing videos on TikTok?

Re:QwQ: Alibaba's O1 Like Reasoning LLM by nextworddev on Wednesday November 27, 2024 @08:59PM (#42261687)
you get video tokens through those seemingly dumb tiktok shorts
Re:QwQ: Alibaba's O1 Like Reasoning LLM by nl on Wednesday November 27, 2024 @10:15PM (#42262099)
Of all the types of tokens in the world video is not the one that comes to mind as having a shortage.

By setting a a few thousand security cameras in various high traffic places you can get almost infinite footage.

Instagram, Youtube and Snapchat have no shortage of data too.

Re:QwQ: Alibaba's O1 Like Reasoning LLM by nextworddev on Wednesday November 27, 2024 @10:20PM (#42262123)
except 1) tiktok is video stream data many orders of magnitude larger than any security cam data, that's attached to real identity 2) china doesn't have direct access to Instagram reels and shorts, so yeah
Re:QwQ: Alibaba's O1 Like Reasoning LLM by nl on Wednesday November 27, 2024 @10:27PM (#42262156)
Why does tying it to identity help LLM training?

It's pretty unclear that having orders of magnitude more video data of dancing is useful. Diverse data is much useful!