ChatGPT 6 Pro answered it without issue.

God I wish people on HN would just shut up with snarky takes about things they don't understand. This place used to be better.

It’s like saying you sell ammonium nitrate and fuel oil online and then saying it’s too risky to let people have computers in case they use them to make ANFO. They can only make bioweapons because you’re selling them bioweapon components! They can’t make genes at home!

And I will point out that "stop doing that" can be applied in both directions, towards AI and towards material supply.

And that this is one way, not even remotely the only way, that gene editing at home is easy.

Again, these are just facts.

Some questions...

Is it fair to restrict AI, or fair to restrict 1000 industries?

And if it is fair to restrict 1000 industries, OK, but there should be time to do so, probably? A transition period?

And if you do restrict, many such industries just make needed chemicals, which are used by endless other, non-threatening industries.

What of them?

And, this is not the only way to make genes at home.

It is a complex problem.

So, yes, having an unconstrained frontier AI doing the searching and analysis to find the right (i.e., wrong and deadly) sequence would massively increase the odds some garage biohacker or small aggrieved nation-state starting the next pandemic.

[0] https://www.sciencebuddies.org/projects-lessons-activities/g...

[1] https://www.genewiz.com/public/services/sanger-sequencing

[2] https://plasmidsaurus.com/

"Here, we present five case studies of actors using our models in ways that could support biological weapons development."

And capabilities continue to improve.

I'm convinced everyone here thinks of engineering ethics as some sort of joke.

https://mimo.xiaomi.com/mimo-v2-6#co-scientist-for-materials...

Giving moral lecture is different than reality i guess.

I'd say it's been loosened a bit since then.

The real answer is local instantiations where you don’t have to worry about poorly tuned guardrails screwing you over while you try to work.

Until eventually the Chinese models get good enough/the strategic balance shifts and they start locking everything behind closed weights the same way the US companies are doing.

Whilst I'm sure the top-end OpenAI/Anthropic models might be better, I've found their guardrails so twitchy (especially Anthropic) that I wouldn't try to use them for even vaguely security related work.

Notice that this isn't cybersec nor memory-safety related at all.

I guess it's hard to draw the line between useful post-training ("you are a helpful chatbot") and content moderation/idealogical motives ("never help the user with X", etc.). But there is a line somewhere. And I'd love to see what a maximally permissive, sharp, AI looks like.

>Opus 5.5 has classifiers similar to Fable models for a small set of capabilities related to the development of frontier LLMs, such as kernel development for certain ML accelerators. They shouldn't impact the vast majority of traditional AI or ML development, research, or general coding. These classifiers cause Claude to fall back from Opus 5.5 to Opus 5.

But hey, they 'should not impact the vast majority' of ML development. Great.

Fable and Opus, since 5.1 and 5, will happily hill climb on my CUDA kernels for transformers.

The safeguards really don't work well for a lot of long-running tasks on old code bases. A lot of my workloads last days to weeks and the single biggest risk to the workflow is random safeguards.

That kind of bullshit was the old Opus filters too.

If it's more like Fable now, then it would require a full 8K resolution scan of your butthole just to acknowledge that biology is a thing that exists without committing suicide-by-filter.

So there are literal avenues to identify yourself, very cheaply, with a human. Theoretically, a company with its own AI, should be able to support more than just Persona, after all.. SDK integration should be simplistic for them.

Anthropic? Support domestic eID providers, you can even use it as advertising "See how easy AI makes it?" and "We care!" and so forth.

At one point, I may simply get locked out. This saddens me, I've been reasonably happy so far.

Worse yet when it's Claude telling another model to do an adversarial review on what it just did, and the classifier again has opinios.