pyridines

> Before approving construction, I would want communities of humans to understand why the design works and what justifies confidence in its safety. I would hope that we all would.

Until very recently, I pored over every single line of code Claude generated with razor sharp scrutiny. I would usually catch issues with every response. I'm catching fewer problems these days. Maybe the model is just getting better, and maybe I'm being less careful while under pressure to ship more and more often. But model capability is obviously growing. Even back in March, you could tell it "give me a function that adds two numbers" and you could be 100% confident that it would write the correct function. There was almost no point in looking at the code. Since then, the complexity floor of problems in the category "this is so simple that the model couldn't possibly get it wrong" is rising, and with it, my cognitive surrender to the model is increasing too. Why check it? It's obviously going to be correct.

If AI designs a terawatt fusion plant, then of course we're going to meticulously pore over every detail to ensure safety, reliability, efficiency, whatever. If we find no flaws in the design whatsoever, will we be less careful about the second one? The third one? What about the ten thousandth one? Will "a nuclear fusion plant" become something that models couldn't possibly get wrong?

Terence Tao is arguing that the human involvement in research is crucial, but doesn't convincingly justify why, in my opinion. He says that "human agency is a value of fundamental importance" and that we will need to build "thriving human communities that can understand [AI ideas] together" - not for the sake of correctness, which AI may surpass us on, but for, I guess, the possibility of reclaiming human meaning and purpose. I don't disagree with this at all, but it's not an argument, it's a statement of values. Unfortunately, the stark reality is that if AI does surpass humans, it will become the economically dominant strategy to not verify them and not double check them, but to just do whatever they say. This seems like a great way to raise p(doom). But as the models get better and better, and as I'm scrutinizing Claude's output less and less... I just hope that there are more Terence Taos out there than people like me.

~

I get what you're saying. The thing itself is just math. I'll just say it depends on how you define intelligence. If at some point we're be able to simulate a human brain with 100% accuracy, I would say that it is intelligent, it sounds like you would not. (I don't mean to imply consciousness or personhood or anything else by "intelligent".)

Okay, I guess you're right that its ability to do this is just correlational, which doesn't imply it has any understanding. However, you have to conclude that some tasks which we used to believe required intelligence don't actually require any, which is disconcerting.

The very fact that it is able to search within a meaning-space demonstrates that it understands semantics, to some extent. Philosophically, that is profound, for something that is just one big matrix multiplication. Drawing connections between things in meaning-space is surely a facet of intelligence.

This looks interesting, but would you mind saying a sentence or two about why before I commit to an hour-long video? It looks like it shows how they work internally, which is sort of a non sequitur. Brains also work mechanistically. I'm claiming that any system which is able to do what AIs do must necessarily have some sort of intelligence.

I just can't accept that it possesses no intelligence. It is not equivalent to human intelligence, obviously, but how can a system without some semblance of rational thinking solve open math problems? Even composing earlier human work into something novel requires intelligence and understanding on some level.

That makes sense, though it sounds like it would still be feasibly cheap to only track a handful of users or organizations

> Can a watermark be traced back to me or my organization?

> No. The watermarking applies to Claude and its outputs. It doesn’t identify anything to do with individual users

But there's nothing stopping them from doing this in the future. In fact, couldn't they append the user's ID to the key in some manner?

The wording in this post seems much more... restrained? than usual. Maybe Anthropic is afraid of exaggerating the capabilities and consequences of their new models to avoid government scrutiny and sanctions.

> we’ve intentionally avoided training Opus 5 on cyber tasks [...] it remains substantially behind Mythos 5 on the exploitation of those vulnerabilities

I wonder if Anthropic would still intentionally nerf their models without the threat of government intervention.

What I hate about my new Toyota's volume knob is that there is no indication of volume level in the UI, and the knob itself doesn't ratchet. So I have absolutely no feedback about how much louder or quieter it's going to get when I turn the knob. If I have no music going, but I'm waiting to hear the next GPS instructions, how can I make sure I'm going to hear them? If I'm not sure where the volume is at right now, I can't, unless I turn it and then try and trigger some sound effect or something. It's needlessly complicated.

> no need to tear them down after a couple of decades

Where are people constructing buildings that need to be torn down after a couple decades? This is a refrain I've heard about construction (in the US?) several times in discussions like these, and I find it very puzzling.

IME it's Claude that pushes back, and Codex that just does the thing. It's happened once or twice where I've told Claude bluntly and directly "do this" and it responded "no, here's why that's a bad idea..." Maybe it's just my CLAUDE.md.

Not sure if there are sycophancy benchmarks for coding agents

An image model could probably generate gore as long as there was, say, both PG-13 violence and surgery photos in the training set. There's probably no way to prevent the ability of the model to generate disturbing imagery without also sacrificing its ability to make acceptable things.

'censorship' may be too strong a word, but there is something unprecedented about this. AI tools are supposed to be general-purpose and able to assist with all sorts of tasks. It's expected that they are restricted when it comes to "unsafe" content like illegal or nsfw information and activities. However, this is the first time, to my knowledge, that an AI tool has been restricted from assisting with something that's perceived as a threat to the AI company.

In my experience, an agent with "fresh eyes", i.e., without the context of being told what to write and writing it, does have a different perspective and is able to be more critical. Chatbots tend to take the entire previous conversational history as a sort of canonical truth, so removing it seems to get rid of any bias the agent has towards the decisions that were made while writing the code.

I know I'm psychologizing the agent. I can't explain it in a different way.

It is ridiculously more expensive and complicated under the hood, technically, but to the user, the sheer convenience of being able to text the computer "hey, when I get an email like X, inform Y and do Z" and that's it, you're done, is unmatched.

Some nitpicking.

> Modern JavaScript applications aren't just a few scripts anymore — they're sprawling codebases with thousands of dependencies, complex module graphs, and extensive build pipelines.

This has been true for at least a decade.

The very next paragraph:

> JavaScript-based tools that were once "good enough" now struggle to keep up, leading to sluggish build times, laggy editor experiences, and frustratingly slow feedback loops.

The tools really weren't "good enough", even back then, by these metrics. JavaScript tooling has been slow and bloated on large codebases for just as long.

Animal intelligence is often underestimated, (e.g. there's a famous test that shows that chimpanzee working memory is better than ours) but our use of language is qualitatively different from other animals. Some animals have rudimentary communication, but no other animal is capable (as far as we know) of recursive, infinitely variable language structure like us.

> spaghettification is also a non-issue, as long as the framework defines clear containers for spaghettis

Sorry, but I disagree strongly with this. When there is, inevitably, a bug that the LLM can't fix, someone's going to have to read all that spaghetti, and they'll curse whoever put it there. "clear containers for spaghetti" is a pipe dream, all abstractions leak, and the bug may very well be in the spaghetti. "Just start over" is unrealistic for large, complex apps.

Of course, if you really have a solution for this, that would be incredible.

they're referring to the infamous Corporate Memphis style which is frequently used in big tech branding: https://en.wikipedia.org/wiki/Corporate_Memphis

It's not unreasonable to briefly forget details like that, especially when you're dealing with a multi-language codebase where "how do I make a log statement?" requires a different pattern in each one.

Another American holiday coming up with an equally useless name is Fourth of July. Nobody seems to have a problem with that name, and nobody I know calls it Independence Day. Neither Fourth of July or Juneteenth are great names out of context, but they both have histories behind them and can't be changed anymore.

Heck, Juneteenth is a better name, since it is not literally month+day.

I was dumbfounded too, but NBC explains in this same article:

> "The law banning TikTok [...] allows the president to grant a 90-day extension before the ban is enforced, provided certain criteria are met."

I agree - if I was a student, I would be very tempted to use an LLM for this sort of coursework-irrelevant busywork assignment, especially if I had other work on my plate. It's not so hard to rationalize using it for this type of thing vs. an "actual" assignment, all due respect to the professor, who I'm sure means well

A shame there's only one doctor left :(

Oh, that's nothing. I present to you a 1 byte program that outputs the complete works of Shakespeare:

a

The interpreter, written in Rust (for performance), evaluates the command `a` by printing to STDOUT a particular payload included in the runtime.

Here's my two cents, not directed towards you, but to the discussion in general.

If someone wants to lease a product to you, and you would rather they sell it, that doesn't give you the right to pirate it. When the only way to buy a thing is to "buy" a misleading lease, piracy becomes more "permissible", as in people will blame you less for pirating it, but that doesn't make it morally good.

It's obviously a bad thing for the publisher to make it seem like you're buying something, then pull the rug out from under you. It's extremely misleading, maybe it should be illegal. It's definitely a bad thing, even if it's all technically there in the fine print that nobody reads. Pirating the thing in response to this is a situation where two wrongs don't make a right. It's debatable, sure, but it's morally gray at best.

No matter how you slice it, you're still depriving the creators/publishers/etc of the money they ask for in return for the thing you want. Maybe you don't like the publisher, maybe you think the platform is evil, but you are still taking what doesn't belong to you.

A big reason it's bad is the plain ol' categorical imperative. If everyone did it, there would be no more music, movies, games, etc, because the people making them wouldn't get paid. That's true regardless of how the thing is sold.

If you don't like how they're selling it, boycott it. That's fine. It's not like we're talking about essential goods like shelter or food. Most of the time, the discussion is around entertainment.

TL;DR Pirating is morally gray at best, but it is not 100% morally good, and I wish people would be honest about that.

I'm reminded of the Itchy and Scratchy clip where they each pull out bigger and bigger guns on each other, until finally the earth explodes.