andreyk

"and we didn't see anything" is not justified at all.

Meta absolutely has (or at least had) a word class industry AI lab and has published a ton of great work and open source models (granted their LLM open source stuff failed to keep up with chinese models in 2024/2025 ; their other open source stuff for thins like segmentation don't get enough credit though). Yann's main role was Chief AI Scientist, not any sort of product role, and as far as I can tell he did a great job building up and leading a research group within Meta.

He deserved a lot of credit for pushing Meta to very open to publishing research and open sourcing models trained on large scale data.

Just as one example, Meta (together with NYU) just published "Beyond Language Modeling: An Exploration of Multimodal Pretraining" (https://arxiv.org/pdf/2603.03276) which has a ton of large-experiment backed insights.

Yann did seem to end up with a bit of an inflated ego, but I still consider him a great research lead. Context: I did a PhD focused on AI, and Meta's group had a similar pedigree as Google AI/Deepmind as far as places to go do an internship or go to after graduation.

This is quite misleading... From the article:

“When the Waymo vehicle encounters a particular situation on the road, the autonomous driver can reach out to a human fleet response agent for additional information to contextualize its environment,” the post reads. “The Waymo Driver [software] does not rely solely on the inputs it receives from the fleet response agent and it is in control of the vehicle at all times.” [from Waymo's own blog https://waymo.com/blog/2024/05/fleet-response/]

What's the problem with this?

To say they LLMs are 'predictive text models trained to match patterns in their data, statistical algorithms, not brains, not systems with “psychology” in any human sense.' is not entirely accurate. Classic LLMs like GPT 3 , sure. But LLM-powered chatbots (ChatGPT, Claude - which is what this article is really about) go through much more than just predict-next-token training (RLHF, presumably now reasoning training, who knows what else).

This blog post is full of bizarre statements and the author seems almost entirely ignorant of the history or present of AI. I think it's fair to argue there may be an AI bubble that will burst, but this blog post is plainly wrong in many ways.

Here's a few clarifications (sorry this is so long...):

"I should explain for anyone who hasn't heard that term [AI winter]... there was much hope, as there is now, but ultimately the technology stagnated. "

The term AI winter typically refers to a period of reduced funding for AI research/development, not the technology stagnating (the technology failing to deliver on expectations was the cause of the AI winter, not the definition of AI winter).

"[When GPT3 came out, pre-ChatGPT] People were saying that this meant that the AI winter was over, and a new era was beginning."

People tend to agree there were two AI winters already, one having to do with symbolic AI disappointments/general lack of progress (70s), and the latter related to expert systems (late 80s). That AI winter has long been over. The Deep Learning revolution started in ~2012, and by 2020 (GPT 3) huge amount of talent and money were already going into AI for years. This trend just accelerated with ChatGPT.

"[After symbolic AI] So then came transformers. Seemingly capable of true AI, or, at least, scaling to being good enough to be called true AI, with astonishing capabilities ... the huge research breakthrough was figuring out that, by starting with essentially random coefficients (weights and biases) in the linear algebra, and during training back-propagating errors, these weights and biases could eventually converge on something that worked."

Transformers came about in 2017. The first wave of excitement about neural nets and backpropagation goes all the way back to the late 80s/early 90s, and AI (computer vision, NLP, to a lesser extent robotics) were already heavily ML-based by the 2000s, just not neural-net based (this changed in roughly 2012).

"All transformers have a fundamental limitation, which can not be eliminated by scaling to larger models, more training data or better fine-tuning ... This is the root of the hallucination problem in transformers, and is unsolveable because hallucinating is all that transformers can do."

The 'highest number' token is not necessarily chosen, this depends on the decoding algorithm. That aside, 'the next token will be generated to match that bad choice' makes it sound like once you generate one 'wrong' token the rest of the output is also wrong. A token is a few characters, and need not 'poison' the rest of the output.

That aside, there are plenty of ways to 'recover' from starting to go down the wrong route. A key aspect of why reasoning in LLMs works well is that it typically incorporates backtracking - going earlier in the reasoning to verify details or whatnot. You can do uncertainty estimation in the decoding algorithm, use a secondary model, plenty of things (here is a detailed survey https://arxiv.org/pdf/2311.05232 , one of several that is easy to find).

"The technology won't disappear – existing models, particularly in the open source domain, will still be available, and will still be used, but expect a few 'killer app' use cases to remain, with the rest falling away."

A quick google search shows ChatGPT currently has 800 million weekly active users who are using it for all sorts of things. AI-assisted programming is certainly here to stay, and there are plenty of other industries in which AI will be part of the workflow (helping do research, take notes, summarize, build presentations, etc.)

I think discussion is good, but it's disappointing to see stuff with this level of accuracy being on front page of HN.

For reference, the details about how the LLMs are queried:

"How the players work

    All players use the same system prompt
    Each time it's their turn, or after a hand ends (to write a note), we query the LLM
    At each decision point, the LLM sees:
        General hand info — player positions, stacks, hero's cards
        Player stats across the tournament (VPIP, PFR, 3bet, etc.)
        Notes hero has written about other players in past hands
    From the LLM, we expect:
        Reasoning about the decision
        The action to take (executed in the poker engine)
        A reasoning summary for the live viewer interface
    Models have a maximum token limit for reasoning
    If there's a problem with the response (timeout, invalid output), the fallback action is fold"

The fact the models are given stats about the other models is rather disappointing to me, makes it less interesting. Would be curious how this would go if the models had to only use notes/context would be more interesting. Maybe it's a way to save on costs, this could get expensive...

But LLMs would presumably also condition on past observations of opponents - i.e. LLMs can conversely adapt their strategy during repeated play (especially if given a budget for reasoning as opposed to direct sampling from their output distributions).

The rules state the LLMs do get "Notes hero has written about other players in past hands" and "Models have a maximum token limit for reasoning" , so the outcome might be at least more interesting as a result.

The top models on the leaderboard are notably also the ones strongest in reasoning. They even show the models' notes, e.g. Grok on Claude: "About: claude Called preflop open and flop bet in multiway pot but folded to turn donk bet after checking, suggesting a passive postflop style that folds to aggression on later streets."

PS The sampling params also matter a lot (with temperature 0 the LLMs are going to be very consistent, going higher they could get more 'creative').

PPS the models getting statistics about other models' behavior seems kind of like cheating, they rely on it heavily, e.g. 'I flopped middle pair (tens) on a paired board (9s-Th-9d) against LLAMA, a loose passive player (64.5% VPIP, only 29.5% PFR)'

haha nice, the official caltrain schedule is a bit of a hassle to parse...

Seems like a good overview, but I do find this bit unclear: "But why don’t market forces correct these issues?

The answer lies in the unique shield that non-dischargeable student loans provide to educational institutions and lenders.

In a normal market, if a product consistently fails to deliver value, consumers stop buying it. Producers either improve or go out of business. But in the world of higher education, this feedback loop is broken.

Colleges and universities, shielded by the guarantee of student loan money, have no real incentive to improve their product or direct students to majors that have an ability to pay back their loans.

They can raise tuition year after year, even as the value of their degrees stagnates or declines. "

Sure, colleges can charge a lot due to loans, but they are still competing with each other and differences in tuition could make a big difference. I went to Georgia Tech over other universities because it was in-state and Georgia has generous scholarships for students with good grades. So why does competition among schools not lower costs?

Exciting to see this so soon after Anthropic's "Mapping the Mind of a Large Language Model" (under 3 weeks). I find these efforts really exciting; it is still common to hear people say "we have no idea how LLMs / Deep Learning works", but that is really a gross generalization as stuff like this shows.

Wonder if this was a bit rushed out in response to Anthropic's release (as well as the departure of Jan Leike from OpenAI)... the paper link doesn't even go to Arxiv, and the analysis is not nearly as deep. Though who knows, might be unrelated.

I have finished a PhD in AI just this past year, and can assure you there exist reviewers who spend hours per review to do it well. It's true that these days it's often the case that you can (and are more likely than not to) get unlucky with lazier reviewers, but that does not appear to have been the case with this paper.

For example just see this from the review of f5bf:

"The main contribution of the paper comprises two new NLM architectures that facilitate training on massive data sets. The first model, CBOW, is essentially a standard feed-forward NLM without the intermediate projection layer (but with weight sharing + averaging before applying the non-linearity in the hidden layer). The second model, skip-gram, comprises a collection of simple feed-forward nets that predict the presence of a preceding or succeeding word from the current word. The models are trained on a massive Google News corpus, and tested on a semantic and syntactic question-answering task. The results of these experiments look promising.

...

(2) The description of the models that are developed is very minimal, making it hard to determine how different they are from, e.g., the models presented in [15]. It would be very helpful if the authors included some graphical representations and/or more mathematical details of their models. Given that the authors still almost have one page left, and that they use a lot of space for the (frankly, somewhat superfluous) equations for the number of parameters of each model, this should not be a problem."

These reviews in turn led to significant (though apparently not significant enough) modifications to the paper (https://openreview.net/forum?id=idpCdOWtqXd60&noteId=C8Vn84f...). These were some quality reviews and the paper benefited from going this review process, IMHO.

"Leadership worked with you around the clock to find a mutually agreeable outcome. Yet within two days of your initial decision, you again replaced interim CEO Mira Murati against the best interests of the company. You also informed the leadership team that allowing the company to be destroyed “would be consistent with the mission.”"

wow, this is a crazy detail

"Leadership worked with you around the clock to find a mutually agreeable outcome. Yet within two days of your initial decision, you again replaced interim CEO Mira Murati against the best interests of the company. You also informed the leadership team that allowing the company to be destroyed “would be consistent with the mission.”"

insane

Altman has been the CEO of OpenAI since early 2019, so he has been there for the whole journey from GPT-2 to GPT-4/ChatGPT and the company's shift to mostly focusing on LLMs over RL/robotics/other stuff. I can't say I know much about the degree of his involvement, but my gut feeling is that he has been heavily involved in steering the company's direction and efforts.

Yeah, that's most likely true. I am less sure this would be a "far future" feature, though, given it's probably not a ton of work and power users would probably be willing to pay for it. We shall see, OpenAI moves fast...

"The purely prompt-driven ones are essentially just ChatGPT in a trench coat. They’re effectively a way of bookmarking and sharing custom instructions, which is fun and useful but doesn’t feel like a revolution in how we build on top of these tools."

This is missing one important aspects of GPTs: fine tuning. As with ChatGPT, the UI allows you to thumbs up / thumbs down replies, which results in data that OpenAI can be used to improve the model. If (and I have no idea if this is the case) OpenAI invests in finetuning individual GPTs on their own distinct datasets, a GPT could diverge from being a "chatgpt in a trenchgoat" pretty significantly with use.

TLDR: "For the first time, I do start to see it. Five billion people is almost 600 New Yorks. Almost 600 future New York Cities will need more water. Everyone I pass on my bicycle (including the climate refugees soon to be sequestered next to the old airstrip) will be thirsty. Everyone on the subway, everyone visiting the Empire State Building, everyone in a Broadway audience — thirsty, if they live in the wrong New York City. And repeat that 600 times. Oddly then, I can see the shape of that number. And 5 billion, along with the parts per million and degrees of warming, starts to make sense. To learn this stuff requires new intuitions. The old ones don’t scale. ... We use the world “unimaginable” a lot, and rarely with optimism. But the world is far, far more imaginable if you see everything as a multiple of home."

Pretty common sense stuff. Personally, I think we need to have a lot more "climate change art". Nothing has driven the reality of how catastrophic climate change will be for me quite like the opening chapter of The Ministry of the Future - https://www.orbitbooks.net/orbit-excerpts/the-ministry-for-t...

Incidentally, I just bought the Gen 1 of these before departing on a trip to Japan a week ago. I gotta say, I'm very happy with the purchase - being able to take photos/videos for memory-capture type stuff (ie I don't care a lot about it being the best quality) without taking out my phone and pointing it at stuff has been great, and the speakers are surprisingly nice to use when I don't want to block out the sounds of my surroundings. To me the benefit is not being able to constantly or secretly record stuff, it's just a nicer way to use a camera to record memories on a trip that does not involve staring at a screen half the time.

Long quote, but I think this is useful context for the argument:

"LLM believers will probably demur: But humans also make mistakes, and surely we’re not prepared to say that humans can’t reason just because they make mistakes? First, it is not accurate to say without qualification that “humans can reason,” certainly not in the sense that we can randomly pluck any person from the street and expect them to reliably perform normatively correct reasoning. Most neurobiologically normal humans have the capacity to become proficient in reasoning, but actually attaining such proficiency takes significant training and discipline. ... But if a human made these mistakes, the ones reported in this article, then I would conclude without any hesitation that they cannot reason. Even if they went on to list a large number of other examples demonstrating impeccable reasoning, I would suspect that other factors (such as rote memorization or cheating) were behind the performance discrepancy. For the mistakes reported here are not performance mistakes, the sort of innocuous errors that humans might make—and promptly correct—when they are careless or tired. If a human made these mistakes, and made them consistently under repeated questioning, that would indicate without doubt that they don’t have the necessary logical competence, that they lack fundamental concepts that are part and parcel of the fabric of reasoning, such as logical entailment and set membership."

So really what this is saying is "GPT-4 makes certain categories of mistakes wrt reasoning that indicate it is in general not doing 'true reasoning' even if it does say the right things to indicate reasoning in other cases". And yeah, if that's the basis of your argument, sure. But how would it be doing rote memorization or "cheating" in the cases it does get things right? A weird notion...

Anyway it feels rather pointless to make this a binary quality. As this article points out, humans (on average) make various reasoning mistakes due to cognitive biases as well. GPT-4 *can* output valid explanations for its reasoning for various questions, but fails to do so correctly in many cases (as shown in this piece), and to me it is more interesting to discuss the implications of this rather than to just establish that fact (which is not news to anyone afaik). This does have a 'Conclusions' section that delves into this a little, but it rather over-general and weak.

Still, this is pretty well written and it is good to have a compilation of examples to demonstrate GPT-4 is still not a "human-like reasoner" for anyone not aware of these models still having such flaws, I suppose.