News for hackers. Stuff that matters.
Re:AI Can Now Design Functional Viruses. Should We Worry? by timr on Saturday August 15, 2026 @12:14PM (#49311776)
The design was never the hard part. You still have to make the thing.

Designing a modifcation to an existing virus is perhaps more achievable, but the implementation is still the hard part.

Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @12:17PM (#49311803)
You can literally send an email to one of hundreds of companies that do exactly this. They have no way of knowing whether you're asking them to synthesize a virus or a drug candidate or what.
Re:AI Can Now Design Functional Viruses. Should We Worry? by timr on Saturday August 15, 2026 @12:39PM (#49312012)
No, you literally cannot. Synthesizing a stretch of DNA is trivial, and what you can send away for [1].

Building a functional virus is not even remotely a turn-key operation. Assuming you could get the genome, expressing that genome in a way that results in infectious virus is non-trivial.

[1] Even then, the companies will rarely give you long enough reads to make a full genome scaffold. It looks like there are two companies that will do 50-100kb assemblies as a service today, but I guarantee you they're going to look at what you're trying to make, and who is asking them to do it.

Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @12:46PM (#49312089)
https://www.genewiz.com/public/services/gene-synthesis/custo...

There you go!

Steps from your mailbox to an actual virus is a few days of bench work.

Re:AI Can Now Design Functional Viruses. Should We Worry? by timr on Saturday August 15, 2026 @12:54PM (#49312173)
Yes, that's one of the ones I was talking about.

That's not a service that makes you a virus, and making one from an mRNA transcript is not trivial.

Re:AI Can Now Design Functional Viruses. Should We Worry? by smallmancontrov on Saturday August 15, 2026 @01:05PM (#49312267)
And one wouldn't make one from mRNA. This guy doesn't know what he's talking about. Still: the difficulty isn't comparable to EUV machines or refining uranium and does not form a comfortable security barrier.
Re:AI Can Now Design Functional Viruses. Should We Worry? by timr on Saturday August 15, 2026 @01:14PM (#49312351)
I generally agree -- the actual risk is from serious/state actors, and for those, the barriers are not high.

I'm just saying that people are focused on the wrong problem because they've got LLM-brain.

Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @01:29PM (#49312464)
The only difference between people who are quite concerned and you is that you are either 1) actually blind to or 2) choosing to ignore even the first derivative on those barriers.

The people "got LLM-brain" are saying: "Wow those barriers are coming down fast"

And you're saying: "There are still barriers"

Both are true, but only one is helpful for thinking about the future. The other is hardly even helpful for thinking about the present, and I've never seen a quasi-intelligent person do it in lieu of some other motivations in their reasoning.

Re:AI Can Now Design Functional Viruses. Should We Worry? by smallmancontrov on Saturday August 15, 2026 @02:25PM (#49312974)
So what's your proposal? Lie about the risks enthusiastically enough that someone else does something useful?
Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @04:03PM (#49313806)
Huh?

You know all you need to do to demonstrate that I’m lying is to share how exactly commercial labs can detect novel pathogens before producing them, right?

You haven’t done that because you can’t, because they don’t.

They claim to compare fragments to known pathogens. That is obviously less and less secure by the day (see: TFA). But even that obvious security measure has already been demonstrated by red team exercises as trivially defeatable.

Get off your high horse.

Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @01:07PM (#49312284)
Here let's just get specific.

For each of the years below, what is YOUR estimate for how long would it take a person of 80th percentile intelligence with $10,000,000 (2026 money) to get from "I'd like to kill a lot of people" to a working virus?

You can provide a range for each if you'd like.

In the year 1800 – how long?

1950?

2020?

2026?

2050?

2200?

Re:AI Can Now Design Functional Viruses. Should We Worry? by rbliss on Saturday August 15, 2026 @12:40PM (#49312019)
Yet.
Re:AI Can Now Design Functional Viruses. Should We Worry? by timr on Saturday August 15, 2026 @12:53PM (#49312162)
No, the sophomoric LLM argument does not apply here. The world is not a big computer.

The point is that the rate-limiting step in producing something infective is a couple million dollars in special lab equipment, skilled practitioners, and incredibly tricky and tedious procedures that take time and repetition to master.

So yeah, you can try to pay someone who has that expertise to make your top-sekret superbug (good luck with that), or you can do what any real threat actor is going to do, and put in the time and investment. That's the hard part, not asking ChatGPT for a genome sequence.

Let me put it another way: despite the synthesis pathways for many common illegal drugs being widely known and trivial enough that an undergrad O-chem major could do them, in fact doing this is a very good way to get caught and arrested. And that's easy.

Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @12:57PM (#49312196)
Your example of how easy it is to control outputs behind specialized knowledge is a thing (recreational drugs) that exist in mass quantities every place on the planet at consumer-accessible prices?
Re:AI Can Now Design Functional Viruses. Should We Worry? by timr on Saturday August 15, 2026 @01:10PM (#49312313)
No. My argument is that many things in the world are hard in a way that computers cannot solve, and the existence of LLMs does not change that.

Synthetic biology is one of those things.

Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @01:17PM (#49312381)
Uhhh right... but all the other hard parts are being solved too, correct?

LLMs are clearly going to solve a part that, 10 years ago, one would've assumed would be one of the highest and last barriers to go down.

Without LLMs, a reasonable security apparatus would be:

Step 1. Disallow dissemination and synthesis of known pathogens

Step 2. Watch closely anyone with the decades of training required to come up with their own pathogen

Re:AI Can Now Design Functional Viruses. Should We Worry? by timr on Saturday August 15, 2026 @02:26PM (#49312985)
> Uhhh right... but all the other hard parts are being solved too, correct? LLMs are clearly going to solve a part that, 10 years ago, one would've assumed would be one of the highest and last barriers to go down.

No.

No, the barriers I'm talking about aren't coming down. Not quickly anyway.

No, nobody assumed this was the "highest and last barrier". That's the point you keep missing.

LLMs are not "solving" the rate-limiting step here. It only looks that way if you don't know what's involved.

Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @04:03PM (#49313808)
Why don’t you just scroll up and answer with your timeline estimates so we can talk about something concrete instead of playing this goofy shell game?

“No no some other step is hard. I won’t say what it is. No it’s not getting easier, miraculously, but I can’t say what it is.”

Re:AI Can Now Design Functional Viruses. Should We Worry? by demibabs on Saturday August 15, 2026 @12:31PM (#49311938)
Would they not be like “hmmm this one looks kinda similar to a known virus”

They really just synthesize anything for you?

Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @12:33PM (#49311952)
I'm sure some of them would, but they're not required by law to do so. And historically the way you'd do this is by actual fragment/substring matches. But these are novel viruses now. So what you really want to do is try to predict whether a novel genome is pathogenic... which is itself obviously a very hard technical problem.
Re:AI Can Now Design Functional Viruses. Should We Worry? by Enginerrrd on Saturday August 15, 2026 @12:56PM (#49312186)
That’s not too much a concern I think. Most of the real pathogenic qualities result from known genes. Inventing a totally novel virus that is also pathogenic is basically science fiction right now. Splicing together known components that could be bad in combination is closer to current technology.
Re:AI Can Now Design Functional Viruses. Should We Worry? by smallmancontrov on Saturday August 15, 2026 @12:43PM (#49312055)
No, of course not. In 2010 I went to a guest lecture from a guy who worked on the scanning algorithm for IDT. A few years later, there was a minor scandal when someone in a nearby lab used a personal credit card for a purchase and got a big shared account locked. So not only were they scanning the content of synthesis orders, they were validating payment back to labs. In 2010, when HTTPS was rare and facebook didn't even offer it.
Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @12:47PM (#49312104)
"There are websites where you can just buy prescription drugs online?"

You: "No, of course not. I once went on a website to buy a drug, and they made go to a doctor and get a prescription and send it to them"

Re:AI Can Now Design Functional Viruses. Should We Worry? by smallmancontrov on Saturday August 15, 2026 @12:54PM (#49312170)
The legit synthesis sites have been scanning and authorizing since before the internet was encrypted. You suggested otherwise, demibabs expressed surprise, and was correct to express surprise because your implication was incorrect.

The black market exists -- but comes with black market problems.

Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @01:01PM (#49312223)
If that's the understanding you came away with, it was clarified in my immediately following comment.

Glad we agree that there are labs that will solve very hard parts of creating a novel virus, either by accident or by negligence.

Re:AI Can Now Design Functional Viruses. Should We Worry? by smallmancontrov on Saturday August 15, 2026 @01:17PM (#49312385)
It was dishonest then for you to imply that readily accessible commercial labs didn't take reasonable precautions and it's dishonest now, too.
Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @01:38PM (#49312538)
Define "reasonable precautions"?

Here's a red team exercise where researchers successfully bought fragments of the best known pathogen in the world, never mind a novel one.

What reasonable precautions failed to catch this but we could expect to catch a novel virus that "kinda looks like" a known virus?

https://www.statnews.com/2024/05/08/shouldnt-be-easy-buy-syn...

Re:AI Can Now Design Functional Viruses. Should We Worry? by smallmancontrov on Saturday August 15, 2026 @02:45PM (#49313173)
So shredding, mutating, and reassembling the genome is the minimum bar to slip it past the scanning? I thought you said you could just order it?
Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @03:57PM (#49313757)
This may have flown past you, but we’re commenting on an article about trivially creating novel pathogens

The shredding, mutating, reassembling is all it took to get the most well known pathogen in the world by the controls that you are suggesting are sufficiently tight

Re:AI Can Now Design Functional Viruses. Should We Worry? by transcriptase on Saturday August 15, 2026 @12:21PM (#49311855)
Yes in much the same way you can literally build a rocket and go to the moon.

You may be skipping a few of the involved steps.

Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @12:25PM (#49311887)
What involved steps are those?

You email the RNA strand (a text file) to a lab along with some money. The lab converts that information into a physical object (read: virus) and sends it back to you in the mail.

What step is missing?

Re:AI Can Now Design Functional Viruses. Should We Worry? by smallmancontrov on Saturday August 15, 2026 @12:45PM (#49312087)
The part where that service doesn't exist and isn't close to existing.
Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @12:47PM (#49312108)
https://www.genewiz.com/public/services/gene-synthesis/custo...

Distance from mailbox to live virus is a few days of bench work from a semi-competent biochemist.

And yes, this service claims to have safeguards. Does every service? Does every future analogous service? Are those safeguards guaranteed to be robust against totally novel pathogens?

Answers: No, no, and no.

Re:AI Can Now Design Functional Viruses. Should We Worry? by smallmancontrov on Saturday August 15, 2026 @12:59PM (#49312203)
Really? mRNA? What kind of meme nonsense is this? You really don't know anything about these protocols do you.

Still, even though your estimates are silly, they aren't silly enough to change the conclusion that this isn't a comforting security barrier, so I suppose we agree on that much.

Re:AI Can Now Design Functional Viruses. Should We Worry? by transcriptase on Saturday August 15, 2026 @08:10PM (#49315557)
I can’t even bring myself to explain how wrong this person is.
Re:AI Can Now Design Functional Viruses. Should We Worry? by consensus1 on Saturday August 15, 2026 @12:23PM (#49311875)
One of the easiest and most effective AI regulations would be to simply require all these labs to run all incoming requests through a suite of the most advanced models asking if the request is likely to be a pathogenic agent.
Re:AI Can Now Design Functional Viruses. Should We Worry? by estearum on Saturday August 15, 2026 @12:27PM (#49311903)
Yes probably true. Unfortunately we need only one defector lab to severely degrade the overall effectiveness of this approach (leaving aside the effectiveness of the detection models themselves).
Re:AI Can Now Design Functional Viruses. Should We Worry? by wat10000 on Saturday August 15, 2026 @12:41PM (#49312028)
Who’s going to simply require that? There are ~200 sovereign nations on the planet, they’re not all going to do it, or have effective enforcement if they do.
Re:AI Can Now Design Functional Viruses. Should We Worry? by ben_w on Saturday August 15, 2026 @12:54PM (#49312172)
This is the kind of thing that would be relatively easy to convince nations to sign up to*; actually enforcing, I suspect you're right.

* there's not an obvious benefit to being the one nation that says "you know what? We want to be known as 'the nation whose biolab insecurity wrecked the world', because taking blame works out so well for everyone.", though I'm fairly sure a large portion of the world will be totally willing to wargame the impact of hosting a lab as a deadman switch so nobody else attacks them first…

Re:AI Can Now Design Functional Viruses. Should We Worry? by dgellow on Saturday August 15, 2026 @12:35PM (#49311972)
That’s one interesting way to generate demand for AI vendors if the AI bet doesn’t pay off /s
Re:AI Can Now Design Functional Viruses. Should We Worry? by haxiomic on Saturday August 15, 2026 @12:22PM (#49311866)
There’s YouTubers who do this [0]. I’d argue estimate the risks of a synthetically induced pandemic of massive impact this century are very high

- [0] https://youtu.be/aoczYXJeMY4?is=OqObeGzZB6AKL0Jk

Re:AI Can Now Design Functional Viruses. Should We Worry? by timr on Saturday August 15, 2026 @12:57PM (#49312192)
Not even close to the same thing. That's taking an existing viral transfection system and swapping in a single gene.

Thermo Fisher sells a kit to do this [1]. It's something you might do a few thousand times on the way to actually generating a novel virus.

> I’d argue estimate the risks of a synthetically induced pandemic of massive impact this century are very high

I'd argue that it probably already happened, and had nothing to do with AI.

[1] AAV transfection.

Re:AI Can Now Design Functional Viruses. Should We Worry? by haxiomic on Saturday August 15, 2026 @05:25PM (#49314471)
A single gene is all that’s needed for human transmission, but multiple genes is not a major step up. This is exactly what I’d expect the most dangerous viruses to be, small gain of function tweaks to existing dangerous viruses, of which there are many to pick from
Re:AI Can Now Design Functional Viruses. Should We Worry? by wat10000 on Saturday August 15, 2026 @12:43PM (#49312058)
“There is another theory which states that this has already happened.”

But the next one (or the first one, if that one wasn’t manmade) will probably be much worse.

Re:AI Can Now Design Functional Viruses. Should We Worry? by 1attice on Saturday August 15, 2026 @01:49PM (#49312641)
You get that the one we had was really really bad for a lot of people, right? A friend of mine buried five members of her immediate family.

I feel like we are all somehow memoryhole-ing this massive trauma that nothing in this batshit era makes any sense without remembering.

Specifically, I wonder if the thirst for a machine god has anything to do with the activation of the Rapture narrative in the collective unconscious, which we then empower by forgetting its trigger. Call Joseph Campbell

Re:AI Can Now Design Functional Viruses. Should We Worry? by wat10000 on Saturday August 15, 2026 @02:56PM (#49313252)
I know it was, but you also get that it wasn't all that bad for most people, right?

It occupied this weird middle ground where it was bad enough to kill a pretty significant number of people, but not so bad that we're all shell-shocked survivors suppressing the trauma of burying loved ones. And nobody wants to characterize it like that. Half the people want to act like it was nothing, and the other half want to act like it was the worst catastrophe to afflict the world in modern times.

I did get pretty suck for a few weeks, recovered. My wife ended up with strange nerve damage that had her using a cane in her 40s for about six months, but she got back to normal. A friend got hit by Long Covid and plans out how to nap during the work day so he can make it to quitting time. But I don't know anybody who died from it. I am NOT saying nobody died from it, just that it wasn't so widespread that everybody had direct experience of death, like would be the case in something like a smallpox epidemic. The US saw over a million dead, which is a hell of a lot, but in a country of 350 million it's not "everybody comes out with massive trauma" a lot.

Plagues get a lot worse than that. Bodies weren't left to rot because nobody was left alive to bury them. Nobody straggled into the next town because it wiped out the entire population of their home town. A few years later, lots of people say with a straight face that it was no big deal. Nobody who went through the Black Death was saying it was no big deal a few years later.

I'm afraid Covid set us up for the next pandemic to be far worse than it has to be, because it was just mild enough that a substantial number of people are able to dismiss it. And that attitude is going to carry over the next time some novel virus starts spreading around the world. And that one might kill half the population instead of <1%.

Edit: and I want to emphasize that this is not a memory-hole thing. In December 2020, one third of Americans polled said that Covid was no more of a threat than the seasonal flu (https://www.ipsos.com/sites/default/files/ct/news/documents/...). A lot of other polls tell a similar story: right in the worst months of the pandemic, a large minority of Americans still thought it was no big deal. That's while it was going on, not in retrospect.

Re:AI Can Now Design Functional Viruses. Should We Worry? by moralestapia on Saturday August 15, 2026 @01:05PM (#49312266)
>the risks of another synthetically induced pandemic of massive impact
Re:AI Can Now Design Functional Viruses. Should We Worry? by hulitu on Sunday August 16, 2026 @12:03PM (#49321247)
> I’d argue estimate the risks of a synthetically induced pandemic of massive impact this century are very high

It was already done. Aparently some 3 letter agencies lack acountability.