chriskanan

Why would one assume larger is needed? I am an AI professor doing work on a wide variety of things but Anthropic models have cost me so much time even when not blocked. My old Anthropic agent wrote rules into its system prompt to avoid all machine learning work. My openai ones do the work but tell me they need to correct my opinions (on fields I helped found) so that they are aligned with the consensus....

Mostly using open models from China in the past two months + Argon.

This is actually wrong. A large fraction of people do not have internal monologues. Not at a computer or I would dig up cognitive science and psychology papers, but language evolved as a means of communication not a means of thought. The vast majority of human thoughts are not verbalized. I have a strong internal monologue, too. I was surprised to learn that a non-trivial number of people do not have one.

I really hate this open weight but closed science approach. These companies just take from academics and all the Chinese companies that are doing good science, but without understanding the recipe it makes it hard to know where the failure points will be until your agent accidentally commits a crime.

Mistral doesn't publish the science.

You are conflating multiple things.

1) First, you are talking about positive forward transfer in continual learning. I've been giving talks for the past 6-7 years about how that community (I was one of the founders) went astray and wasn't focusing enough on that topic, but continual learning of the kind you are thinking isn't in any of these systems right now. I think some people left the Grok team to make a start-up to focus on that. By forward transfer, what I mean is weights update over time and past learning improves future learning such that we get better sample efficiency.

2) Psychologists distinguish among different kinds of intelligence for Spearman's g (IQ). Crystalized intelligence is using already acquired knowledge (frontier models probably have maxed out that). Fluid intelligence is reasoning and finding solutions in novel situations or without the necessary crystalized knowledge. [Giving colloquial definitions]

3) Now, interestingly, neither of those are correlated with _creativity_ (just they are independent, note some have this threshold theory but it hasn't held up in recent papers). That's what the AI's really are terrible at -- creativity. But I'd argue the vast majority of humans aren't very creative, with truly out-of-the-box ideas. Given that this is HN, and a non-trivial number of us have ADHD, creativity is positively correlated with ADHD.

I did a bunch of research on these topics for my AGI course that I teach each Spring (where I then point out conflicting definitions and start using multiple alternative terms rather than AGI to distinguish among the different definitions).

I'm an AI professor, and the same thing happened to me. I had written out contracts and specs for some novel algorithms to compare. And instead it created a new sandboxed environment with 20k lines of code to ensure we do gold standard science.... didn't implement the algorithms at all. In another case, I left it overnight with a plan, and it decided to write a 50k line new overly engineered plan. Suffice to say, I don't use any frontier models for doing coding work anymore. I use them to make detailed plans and use good worker AIs for execution, which don't overthink everything.

Same thing happened with eggs in the USA with bird flu used as an excuse.

And all those reports of Claude when asked without a system prompt what its name was in Chinese it often would say Qwen or Deepseek, etc. I'd love Anthropic to say they aren't distilling and taking from every model out there, because I'm sure they are. As my mom would say, "the pot calling the kettle black." At least Alibaba and other Chinese companies are giving back to the AI community with detailed scientific papers on how their systems work and releasing open-weight or opensource models. I believe Anthropic has released nothing, and given that they had originally configured Fable to sabotage ML related work because only they can be trusted to do it safely, is just anti-science and anti-aligned with what I would consider good human values. They are way too sanctimonious and I don't trust them at all.

They are doing ta tremendous amount of novel research where American AI companies have "war rooms" to study their papers and models and American labs publish next to nothing. They have to often do more with less. As an AI researcher, Chinese labs are doing tremendous benefit to science whereas some American companies (and I'm American) seem to think only they are able to do AI research responsibility (I've been working on neural networks for 25+ years). I'm pretty sure Fable sabotaged my research codebase (see the news stories about this).

Did you ever watch Star Trek: The Next Generation? The current trajectory is like the Ship's Computer. It know everything humanity has learned and can do a lot. But it can't explore and lacks desires and agency. That's why they made a big deal about the character Data being an entirely new kind of AI. Of course Star Trek has a very different economic system and there is a book called Trekenomics about that. So optimistically people live for themselves and don't persue labor they despise. Half of Americans hate their jobs and live for the dream of retirement when they get to actually do what they want.... But they don't have the same energy anymore.

Jobs are an invention of humanity. About 50% of people dislike their job. People spend much of their lives working. Poverty and inequality are a choice made by society if society chooses poorly.

Slightly more nuanced in that the reciprocal reviewer may have been essentially forced to sign despite having other commitments or may not have even been the lead contributor. Nowadays if a student submits a side project to a top-tier conference then it is required that if any authors have significant publication count in top-tier venues, then one must be a mandatory reviewer. Then one must sign that agreement. Students need to publish, much less so for me, where I really want to publish big innovations rather than increments, but now I get all these mandatory reviewer emails demanding I review for a conference because a student has my name on the paper and I'm the most senior, but I may have just seeded the idea or helped them in significant ways. However, many times those are not my passion projects and is just something a student did that I helped with, but now all AI conferences are demanding I review or hurt a student, where I'm the middle author.

But if anything, I think the whole anti-LLM review philosophy is wrong. If anything we need multiple deep background and research analyses of papers. So many papers are trash or are publishing what has already been done or are missing things. The volume of AI papers makes it impossible for a human alone to really critique work because hundreds of new papers come out a day.

I had lunch with Yann last August, about a week after Alex Wang became his "boss." I asked him how he felt about that, and at the time he told me he would give it a month or two and see how it goes, and then figure out if he should stay or find employment elsewhere. I told him he ought to just create his own company if he decides to leave Meta to chase his own dream, rather than work on the dream's of others.

That said, while I 100% agree with him that LLM's won't lead to human-like intelligence (I think AGI is now an overloaded term, but Yann uses it in its original definition), I'm not fully on board with his world model strategy as the path forward.

I can see some promise with diffusion LLMs, but getting them comparable to the frontier is going to require a ton of work and these closed source solutions probably won't really invigorate the field to find breakthroughs. It is too bad that they are following the path of OpenAI with closed models without details as far as I can tell.

This has been a major problem in hospitals where there are many false alarms and often concurrent alarms for hospital beds: https://pmc.ncbi.nlm.nih.gov/articles/PMC3928208/

Same here. I’m an AI professor, but every time I wanted to try out an idea in my very limited time, I’d spend it all setting things up rather than focusing on the research. It has enabled me to do my own research again rather than relying solely on PhD students. I’ve been able to unblock my students and pursue my own projects, whereas before there were not enough hours in the day.

I think we need to distinguish among kinds of AGI, as the term has become overloaded and redefined over time. I'd argue we need to retire the term and use more appropriate terminology to distinguish between economic automation and human-like synthetic minds. I wrote a post about this here: https://syntheticminds.substack.com/p/retiring-agi-two-paths...

See this study, which is consistent with your thesis: https://www.eurekalert.org/news-releases/814485

Essentially, it claims that modern humans and our ancestors starting with Homo habilis were primarily carnivores for 2 million years. We moved back to an omnivorous diet starting around 85,000 years ago after killing off the megafauna, is the hypothesis.

That's the practical reason for why one might care. Keep in mind that the solar system is rotating around the galaxy, so over time different stars become closer or farther away.

As the Kurzesagt video points out, a supernova within 100 light-years would make space travel very difficult for humans and machines due to the immense amount of radiation for many years.

Still, I think the primary value is in expanding our understanding of science and the nature of the universe and our location within it.

A Type II supernova within 26 light-years of Earth is estimated to destroy more than half of the Earth's ozone layer. Some have argued that supernovas within 250-100 light-years can have a significant impact on Earth's environment, increase cancer rates, and kill a lot of plankton. They can potentially cause ice ages and extinctions. Within 25 light-years, we are within a supernova's "kill range." Fortunately, nothing should go supernova close to us for a long time.

Wikipedia article: https://en.wikipedia.org/wiki/Near-Earth_supernova

Kurzgesagt video on the impact on Earth of supernovas at varying distances: https://www.youtube.com/watch?v=q4DF3j4saCE

Read the paper. The media is not providing a lot of missing context. The paper points out problems like leadership failures for those efforts, lack of employee buy-in (potentially because they use their personal LLM), etc.

A huge fraction of people at my work use LLMs, but only a small fraction use the LLM they provided. Almost everyone is using a personal license

This is so shortsighted. The US needs a huge increase in its electricity generation capabilities, and nowadays, rewnewables, especially solar, are the cheapest option.

This video from a few days ago analyzes the issue: https://www.youtube.com/watch?v=2tNp2vsxEzk

Regardless of climate change issues, the anti-renewable policy doesn't seem to make any sense from an economic, growth, or national security standpoint. It even is contrary to the anti-regulation and pro-capitalism _stated_ stance of the administration.

That's my assessment of the report as well.... really, some news truly is "fake" where they are pushing a narrative that they think will drive clicks and eyeballs, and the media is severely misrepresenting what is in this report.

The failure is not AI, but that a lot of existing employees are not adopting the tools or at least not adopting the tools provided by their company. The "Shadow AI economy" they discuss is a real issue: People are just using their personal subscriptions to LLMs rather than internal company offerings. My university made an enterprise version of ChatGPT available to all students, faculty, and staff so that it can be used with data that should not be used with cloud-based LLMs, but it lacks a lot of features and has many limitations compared to, for example, GPT-5. So, adoption and retention of users of that system is relatively low, which is almost surely due to its limitations compared to cloud-based options. Most use-cases don't necessarily involve data that would be illegal to use with a cloud-based system.

Where is the actual paper that makes these claims? I'm seeing this story repeated all over today, but the link doesn't actually seem to go to the study.

I am not going to trust it without actually going over the paper.

Even then, if it isn't peer-reviewed and properly vetted, I still wouldn't necessarily trust it. The MIT study on AI's impact on scientific discovery that made a big splash a year ago was fraudulent even though it was peer reviewed (so I'd really like to know about the veracity of the data): https://www.ndtv.com/science/mit-retracts-popular-study-clai...

Sam Altman way oversold GPT-5's capabilities, in that it doesn't feel like a big leap in capability from a user's perspective; however, the a idea of a trainable dynamic router enabling them to run inference using a lot less compute (in aggregate) to me seems like a major win. Just not necessarily a win for the user (a win for the electric grid and making OpenAI's models more cost competitive).

If they are going to do this, they really ought to corroborate the face recognition with fingerprints. Many people have unrelated doppelgangers, even if an AI algorithm was near perfect: https://twinstrangers.net/

Is there a list of the papers that were flagged as doing this?

A lot of people are reviewing with LLMs, despite it being banned. I don't entirely blame people nowadays... the person inclined to review using LLMs without double checking everything is probably someone who would have given a generic terrible review anyway.

A lot of conferences now require that one or even all authors who submit to the conference review for it, but they may be very unqualified. I've been told that I must review for conferences where some collaborators are submitting a paper and I helped, but I really don't know much about the field. I also have to be pretty picky with the venues I review for nowadays, just because my time is way too limited.

Conference reviewing has always been rife with problems, where the majority of reviewers wait until the last day which means they aren't going to do a very good job evaluating 5-10 papers.