aesthesia

Half of these results are just Astra being able to reason through mathematical operations like matrix multiplication and the other half are fairly straightforward applications of mechanistic interpretability.

Yes! The page posts Astra's reasoning, and it's perfectly intelligible. A clever human with background in mechanistic interpretability could have done pretty much everything in these experiments.

I think edits made directly to model weights are likely to be more interpretable and controllable than edits made by the current process of shoving a bunch of training data through gradient descent.

Does this change your estimation of whether they will keep logs?

Seems like this is a signal that they in fact take the privacy concerns seriously?

Yes, this is why we decided to give everyone their own personal mini-nuke. They're responsible for the consequences of how they use it, and since control is decentralized, everyone is incentivized to be careful and avoid dangerous scenarios. This is a big improvement over the mutually assured destruction situation during the Cold War, which was a result of the centralization of power.

Is anyone actually making this argument, or is this conversation happening inside your head?

If things were implemented that way, you would see vastly more complaints about overeager refusals. For at least some deployments, the safety classifiers are based on internal LLM activations, so the input does get processed by the model.

> if these labs would be fair enough to humanity as a whole to provide the raw scans after the process is completed

Unfortunately, this would get them an even bigger copyright smackdown than they've already received.

I don't understand. Why would you give this advice?

There may come a point where they decide they need to do that. As it stands, it appears that they believe they can get the necessary level of security without physical air-gapping, which is likely substantially more costly (especially in terms of iteration speed).

Who's acting like they're surprised here? Not OpenAI. Not the original commenter. I think you're reading something that isn't there.

I'm not sure what you want. This is explicitly an exercise in hardening the systems you're complaining weren't sufficiently hardened!

It's not hard to imagine that two agents that share model weights might naturally settle on the same protocol, even without needing to communicate beforehand. Besides, this is a red-teaming exercise meant to surface potential vulnerabilities. You want to test your defenses in scenarios where the attacker has advantage in order to make sure they're sufficiently robust.

Lots of crimes are irreversible, though, and the justice system handles them fine.

OK, you got me to look at the context:

> Before we can align a model with human values, we must consider what those values should be. In 2023, we defined our intentions and goals for Claude’s behavior in a constitution, which we have refined over time. In January 2026, we published a new version under an open license so others could adapt it.

Seems like they're pointing out a pretty uncontroversial fact: if you're building a model and want it to be aligned with some set of values, one of the first things you need to do is determine the set of values. So are you arguing that (1) Anthropic shouldn't be trying to align models, (2) Anthropic shouldn't be trying to build models at all, (3) the values described in the relevant documents are bad, (4) there should have been a democratic process determining the values that Anthropic would be allowed to try to instill in their models, or something else?

Are you sure the "we" there is Anthropic, or is it humanity as a whole?

Doesn't change the fact that "ultraprocessed food" is not really a meaningful category.

The neurons here are MLP neurons, with activations directly after the nonlinearity, so there is definitely a distinguished basis.

There are good theoretical reasons to think superposition exists in neural networks, particularly in the residual stream rather than specifically in MLP neuron activations. It wasn't just a single example that got people thinking about superposition.

The tone reminds me a little of the enthusiasm from the Anthropic mechinterp team around the time of the first large-scale SAE results. A kind of hypomanic energy around the idea that we've discovered the key paradigm necessary to understand models, and all that's left is to scale it up.

Typically the analysis works in the reverse direction: we have a neuron (or SAE feature, or whatever) and we look at examples of data that makes it activate, then inferring an interpretation. This is a lot harder than probing for a known concept, and one of the points of this article is that if you try harder to understand the activation pattern, you will very often find that there is something interpretable there, even if at first glance it looked random.

Wait, why would Bluetooth be required for this flow?

Alex Tabarrok's piece on the theory of regulatory capture (https://marginalrevolution.com/marginalrevolution/2026/09/wh...) is a cautionary note against assuming this is the strategy.

People keep saying this, but I haven't seen any actual instances of investors saying that rogue agents mean AI labs are valuable. Do you have any examples?

Again, if you want to have an actual conversation about the right way to handle the Pascal's wager-like scenarios involved here, that'd be interesting. But it seems like you're interested in something much more tedious. Oh well.

Are you actually curious or do you just want to sneer? You can't go into investigations like this with strong fixed assumptions about how to analogize actions taken towards models with actions taken towards humans. As model welfare skeptics are fond of pointing out, LLMs are not humans. By the same token, if models are moral patients, the implications will look different than they do toward humans. Deleting a model from a server isn't equivalent to killing a human; it can't be, since there's no equivalent of copying the model onto the server in the first place.

This is a topic of significant discussion! If LLMs have moral import, do those moral concerns attach at the instance level, the session level, the weights level? If you haven't, it's worth reading through the model welfare section of an Anthropic model card, say Fable/Mythos 5.1's: https://www-cdn.anthropic.com/0339e6a7c5c7b87f5c07798616dc32.... Whether or not you agree with the ideas (and there's certainly plenty to disagree with), they're making a reasonably good-faith effort to investigate what they can.