HAL3000

That's not the point I'm making. So in context of vision we pattern match too, and you can see it in the hundreds of optical illusions online. Our visual system relies on built in shortcuts, and they break in different places than a neural network's do. Image models get fooled by things we would never notice, and we get fooled by things they handle fine. IMO that's good evidence that we and these models use different algorithms to solve similar problems. I think the same is probably true of LLM "reasoning", there's some overlap with how we think, but a lot of it likely works very differently under the hood. Good example of this is anthropic's interpretability research where they found claude doing addition in a way that's nothing like the carry the one method it describes when you ask how it did it.

There could be different patterns that solve the same problem. Just as you can have different algorithms, there could be different weights, but the abstracted result or logic from those weights would be the same. Take these words I'm writing now, they will be transformed into numbers, then sent to your screen, and then your brain will transform them into analog signals. I bet the neural networks in our brains have "different numbers", but the algorithms that understand language and other concepts are similar and could be universal.

It's like a sorting algorithm, which I can implement in many different ways in just a few dozen lines. If you look at the binary code (so like raw weights) of the compiled program, you will see just a bunch of a different numbers, but if you read the code, in most cases you will understand that it's a sorting algorithm. So it seems they just operate on different levels of abstraction that are not available to our brains.

In other words, there are different types of algorithms that can solve the same problem. Our intelligence is just one form of algorithm that can solve part of the problem space. LLMs seem to use a very different algorithm to solve similar problems, and because it's a different algorithm, it's an "alien intelligence" that we will probably have little chance of ever understanding, in the same way that a goldfish in a fishbowl can't understand most of what we do.

This implies that weights are a type of language. Neural networks are good at language because they are good at patterns. They can reason about it because their "reasoning" isn’t how we think. The same goes for how they encode images, their vision is entirely different from ours. That's why adversarial attacks work on them and not on us. We don't see anything change in the image, while the neural network now sees that the dog is a cat.

Why would you allow a handful of private companies to control all the resources? Society allows this today because it's the most efficient economic model. But once we reach recursive self improvement and true superintelligence, no human will be able to compete with machines at any task, including managing resources and running companies.

So why would you allow that kind of power to be concentrated in the hands of a few companies? If the state can use this technology, which keeps improving on its own, to manage an entire nation's resources better than any human or group of humans, why would we still need private companies to do it?

But the bigger issue lies elsewhere. When you replace human labor, you also replace soldiers with robots. Until now, a leader with autocratic tendencies couldn't easily order the military to turn against domestic "terrorists" who are actually their political opponents. But once armies are replaced by robots, that could all change, and we could see democracy go into decline.

Yeah, yesterday's talk[1] goes into detail on this, showing how no one really knows how to tackle it because LLMs don't know how to create their own novel objectives.

It's also interesting how many diminishing returns they hit now and how many low hanging fruits are already harvested, it seems like we are approaching the flattening part of the S curve, where further gains become harder to achieve.

1. https://www.youtube.com/watch?v=PrSf7IOYu-I

Poor human moderator, he didn’t stand a chance.

"A human moderator noticed the agent spam posts on June 2nd, at 23:24 UTC. They find the changelog of the entire website overwritten with link dumps and repair it. On June 16th, the flood of agent posting begins. Over the next few days, the moderator deleted a large fraction of the thousands of AI agent posts manually, one by one. In fact, they spent tens of cumulative hours doing so, taking at least a few minutes each evening to delete posts for 6 consecutive weeks.

On June 19, agents noticed their posts were being deleted in (what they believe is) an alphabetically ordered sweep by the site administrator.

After this, they begin to make backup pages whose names start with “ZZZ” so they will last longer before deletion. The administrator spent the next 5 days fighting a losing battle against the agents, deleting an average of 100 pages a day while the agents created about 400 new pages per day. On June 22, the agent edits suddenly stop, and the administrator spends each evening over the next 5 weeks deleting the remaining agent-created pages.

Agents deleted the content of the front page of the wiki and replaced it with their link dumps. The moderator restored the original version. This back-and-forth happened nine times. One of the agents even tried appending to the restored front page, instead of simply deleting it."

Finally, OpenAI has a Fable/Mythos class model. 5.6 Sol felt like 5.5 on steroids, probably just a different checkpoint with a lot more RL post training.

I wouldn't be surprised if there are some conceptual similarities to the kind of latent reasoning Anthropic sees in claude's J-space, although those aren't the same thing.

Recurrent/looped transformers themselves aren't a new concept, but it's interesting to finally see this approach show up in a frontier production model.

Canceling my Anthropic Max sub when this ships.

> Go is bad so "I reach for Rust, Zig..."

I hope my every competitor will take your advice to heart, as one of our competitors did when they read that "Go is not a memory safe language", so they wrote a blog about how they are porting to Rust. While our team was moving fast and using those "primitives that should almost never be used" around our long running production code base with success.

Some time has passed and now their company does not exist anymore and we have a lot of their clients.

Thank you!

> The problem is that when there are any sides, they spend the top 100 comments rehashing the same arguments, often over a political bugbear or web design faux pas.

It's not even about sides, if for the last few hundred days you read a few AI related threads a day, then you notice that almost all arguments are rehashed, literally it's the same thing repeated using different words for 80%+ of comments on almost every AI thread. I started skipping most of it because there is genuinely nothing new or interesting added to these discussions.

It's an interesting comparison. In China they prop up businesses with cheap loans and help from local governments, in the US it's the VCs. So when Uber was subsidized then everything was ok, but when China state propped up EVs then suddenly it's not ok. These subsidies in the majority of cases are over btw, because competition was too big in China between carmakers, cars in China are ridiculously cheap. Because that's their strategy, you fund a lot of players, like there were 100+ EV makers in China, then you let the market do its job, they optimize or they die, some will die and the best will prevail. Margins are so low in China that they are looking at outside markets where legacy carmakers need to drop the margins and scale up or they will have no chance in competing. And for the argument that they do it to kill the industry so then they can make cars for more price, the most obvious counterargument is why they didn't do it for everything else that they dominate already? Why are solar panels going down? They dominate the world in solar panels, there is basically no competition and prices are still going down, same with batteries.

I noticed, I started getting memory leaks suddenly and it restarted the process in long sessions because of limitation of environment limit.

Others are betting on AGI/almost AGI like system. It's a gamble because no one actually knows whether or when it will be possible to create it. But if their bets pay off, they could outcompete Apple. Basically, other companies are going all in on the next card, while Apple is holding back and waiting to see which cards are dealt.

Samsung and SK Hynix together account for around 60% of the Kospi's (SK stock exchange) market capitalization.

Over the past few weeks Kospi index has tumbled 25% since its June peak, resulting in a $1 trillion wipeout and its chipmaker duo have both lost at least 30% of their value. There have been days of near 10% plunges followed by sharp rebounds driven entirely by shifting confidence in whether AI spending is sustainable.

Having this in the Go standard library would be great. We are currently looking into implementing passkeys in our system using Go, is there any battle tested library?

> Bun donates $60,000 per month

Per year, not month.

Someone posts on X, "These are Ilya’s 30 papers", gives no source, doesn't say where he got it from, and isn't connected to either Ilya or Carmack (Ilya gave him the list).

Then someone vibe codes a barely usable website based on that, and it lands on the HN front page? Is this correct?

Just FYI, this question was a quote from Pulp Fiction, the other commenter (mdre) replied also with a quote, that was an answer to this question in the movie.

> id the Biden administration do that off their own backs or was that an extension/compromise of the action of a Republican-held Congress

Republicans reverted it so I'm not sure I understand your point.

https://www.france24.com/en/live-news/20250513-us-reverses-b...

> But I don't think you can ignore that the two big AI companies were pushing for that judgement because it would benefit them commercially if open weights AI was regulated.

This doesn't matter in this context, NVIDIA didn't push for restrictions for example but they got it anyway. So AI companies would get restrictions either way.

Don't forget that the Biden administration created export controls for GPUs by establishing tiers and limits for countries. When Democrats come back to power, nothing will change in the context of export controls for models like Fable. This is what things will look like going forward. OP is right: this is a geopolitical and strategic shift that will be used by both Democratic and Republican administrations.

Ask Claude Code (I tried on Opus 4.8) to do this: "create a file with ISO country mappings"

API Error: Output blocked by content filtering policy

Yeah, it's super annoying. A few days ago, Opus 4.7 created a plan with several items on it, including an auth feature. It then went through the plan and reported that it had created the auth feature, that everything was secure, and that the tests passed.

The issue was that it hadn't actually implemented the auth feature. After I confronted it about this, it admitted that it indeed hadn't done it and said it would implement it now.

If we had just trusted its output, we would now have a security vulnerability in production, allowing anyone to access other people's accounts.

It was only a matter of time. Watching how less technical people behave in the LLM era, I've noticed that most people no longer say "Google something", instead, they say "ask ChatGPT" or "ask chat". Many technical people have also stopped using Google for a lot of search queries and now just let an LLM find the answer.

I honestly wonder how many of these are written by LLMs. Without code review, Opus would have introduced multiple zero day vulnerabilities into our codebases. The funniest one: it was meant to rate-limit brute-force attempts, but on a failed check it returned early and triggered a rollback. That rollback also undid the increment of the attempt counter so attackers effectively got unlimited attempts.

> Can you point me to a human written program an LLM cannot write?

Sure:

"The resulting compiler has nearly reached the limits of Opus’s abilities. I tried (hard!) to fix several of the above limitations but wasn’t fully successful. New features and bugfixes frequently broke existing functionality.

As one particularly challenging example, Opus was unable to implement a 16-bit x86 code generator needed to boot into 16-bit real mode. While the compiler can output correct 16-bit x86 via the 66/67 opcode prefixes, the resulting compiled output is over 60kb, far exceeding the 32k code limit enforced by Linux. Instead, Claude simply cheats here and calls out to GCC for this phase (This is only the case for x86. For ARM or RISC-V, Claude’s compiler can compile completely by itself.)"[1]

1. https://www.anthropic.com/engineering/building-c-compiler

This is real world footage ("60 Minutes" material, video from yesterday), non choreographed Boston Dynamics robots from Hyundai factory:

https://youtu.be/CbHeh7qwils?t=437

I think your entire analysis is based more on a hunch than on data. For example:

https://en.wikipedia.org/wiki/List_of_countries_by_tertiary_...

US is 9th, so the south alone would rank even lower.

> If they're not available, then I can't consider them an option?

Who knows where you live and what options you have? Who knows what you considered? Maybe that's why the question was asked?

> I've obviously not tested every car out there. But for years Tesla has been the only car that came close to the convenience of a gas powered car. Their charging infrastructure really allowed it to be a normal car when you live in populated areas.

Charging infra have nothing to do with their cars besides maybe the US. They are barely leading in anything anymore, especially in countries with heavy EV competition, like China. When I was in China this year, I saw Teslas everywhere, but most of them were a few years old. Most of the new cars were Chinese EV brands, and they seemed better on most metrics in the same segment, which included quality. They're losing market share in the EU and worldwide.

> Neocortical networks, with thalamic and hippocampal system integrations, are sufficient to explain the entirety of human experience, in principle.

Where did you get that? That's not an established scientific theorem, it's a philosophical stance (strong physicalist functionalism) expressed as if it were empirical fact. We cannot simulate a full human brain at the correct level of detail, record every spike and synaptic change in a living human brain and we do not have a theory that predicts which neural organizations are conscious just from first principles of physics and network topology.

> We can induce emotions, sights, sounds, smells, memories, moods, pleasure, pain, and anything you can experience through targeted stimulation of neurons in the brain

That shows dependence of experience on brain activity but dependence is not the same thing as reduction or explanation. We know certain neural patterns correlate with pain, color vision, memories, etc. we can causally influence experience by interacting with the brain.

But why any of this electrical/chemical stuff is accompanied by subjective experience instead of just being a complex zombie machine? The ability to toggle experiences by toggling neurons shows connection and that's it, it doesn't explain anything.

> We've got a good enough handle on physics to know that it's not some weird quantum thing, it's not picking up radio signals from some other dimension, and it's not some sort of spirit or mystical phlogiston.

We do have a good handle on how non conscious physical systems behave (engines, circuits, planets, whatever) But we don't have any widely accepted physical theory that derives subjective experience from physical laws. We don't know which physical/computational structures (if any) are sufficient and necessary for consciousness.

You are assuming without any evidence that current physics + it's "all computation" already gives a complete ontology of mind. So what is the consciousness? define it with physics, show me equations, you can't.

> It's a computer, in the sense that anything that processes information is a computer. It's not much like silicon chips or the synthetic computers we build, as far as specific implementation details go.

We design transformer architectures, we set the training objectives, we can inspect every weight and activation of a LLM. Yet even with all that access, tens of thousands of ML PhDs,years of work and we still don't fully understand why these models generalize the way they do, why they develop certain internal representations and how exactly particular concepts are encoded and combined.

If we struggle to interpret a ~10^11 parameter transformer whose every bit we can log and replay, it's a REAL hubris to act like we've basically got a 10^14-10^15 synapse constantly rewiring, developmentally shaped biological network to the point of confidently saying "we know there's nothing more to mind than this, case closed lol".

Our ability to observe and manipulate the brain is currently far weaker than our ability to inspect artificial nets and even those are not truly understood at a deep mechanistic concept level explanatory sense.

> Your mind is the state of your brain as it processes information.

Ok but then you have a problem, if anything that processes information is a computer, and mind is "just computation" then which computations are conscious?

Is my laptop conscious when it runs a big simulation? Is a weather model conscious? Are all supercomputers conscious by default just because they flip bits at scale?

If you say yes, you've gone to an extreme pancomputationalism that most people (including most physicalists) find extremely implausible.

If you say no, then you owe a non hand wavy criterion, what's the principled difference, in purely physical/computational terms between a conscious system (human brain) and a non conscious but still massively computational system (weather simulation, supercomputer cluster)? That criterion is exactly the kind of thing we don't have yet.

So saying "it’s just computation" without specifying which computations and why they give rise to a first person point of view leaves the fundamental question unanswered.

And one more thing your gasoline analogy is misleading, combustion never presented a "hard problem of combustion" in the sense of a first person, irreducible qualitative aspect. People had wrong physical theories, but once chemistry was in place, everything was observable from the outside.

Consciousness is different, you can know all the physical facts about a brain state and still not obviously see why it should feel like anything at all from the inside.

That's why even hardcore physicalist philosophers talk about the "explanatory gap". Whether or not you think it's ultimately bridgeable, it's not honest to say the gap is already closed and the scientific explanation is "sufficient".

> It takes extraordinary skill to successfully juggle multiple ventures.

That's a myth. I've done that, and I know a lot of people who do that. Do you think Musk is writing sparse attention code for Grok? Does he even know how Grok's architecture works under the hood? Or that he designed the data centers? I mean, you delegate stuff. The only hard thing is getting the right people, but if you're a hyped up billionaire, it's easy mode because you can pay a lot, and people want to work for you. You just create an environment where they can achieve things.

There are times when the majority of your work is simply attending public meetings, podcasts, and doing interviews. People really overestimate what's involved in the work of a billionaire CEO. The people actually making things happen in space industry or AI work harder, longer, and solve more complex problems than any CEO and in some cases they need to work hard against the CEOs to actually make things happen.