News for hackers. Stuff that matters.
Re:Pause Giant AI Experiments: An Open Letter by root_axis on Wednesday March 29, 2023 @06:12PM (#35364563)
Will it though? Assuming it's even possible for a LLM to e.g. design a novel virus, actually synthesizing the virus still requires expertise that could be weaponized even without AI.
Re:Pause Giant AI Experiments: An Open Letter by tremon on Wednesday March 29, 2023 @06:43PM (#35364929)
Hey GPT-6, construct a floorplan and building instructions for constructing a bioprocess production facility. The building should look like a regular meat packing plant on the outside, but have multiple levels of access control and biohazard management systems.
Re:Pause Giant AI Experiments: An Open Letter by root_axis on Wednesday March 29, 2023 @08:06PM (#35365804)
Let me guess, AI drones to harvest and process the raw materials, construction bots to build the facility, which is of course a fully autonomous bio lab.
Re:Pause Giant AI Experiments: An Open Letter by staunton on Thursday March 30, 2023 @05:25AM (#35370088)
More like Aum Shinrikyo but with an AI as evil mastermind, with brainwashed humans doing its bidding
Re:Pause Giant AI Experiments: An Open Letter by tr_user on Wednesday March 29, 2023 @06:43PM (#35364915)
You're saying they can't make one now. The question is what are we doing before that happens because if you're only thinking about acting when it's viable we're all probably already dead.
Re:Pause Giant AI Experiments: An Open Letter by Teever on Wednesday March 29, 2023 @06:41PM (#35364902)
What if you ask the LLM to design a simplified manufacturing process that could be assembled by a simple person?

What if you ask the LLM to design a humanoid robot that assemble complex things, but could be assembled by a simple person?

Re:Pause Giant AI Experiments: An Open Letter by root_axis on Wednesday March 29, 2023 @08:12PM (#35365859)
LLMs aren't magic, the knowledge of how to design a humanoid robot that can assemble complex things isn't embodied in the dataset it was trained on, it cannot probe the rules of reality, it can't do research or engineering, this knowledge can't just spontaneously emerge by increasing the parameter size.
Re:Pause Giant AI Experiments: An Open Letter by wizzwizz4 on Wednesday March 29, 2023 @06:45PM (#35364947)
I could synthesise this theoretical virus the computer spat out, that may or may not be deadly (or even viable). Or I could download the HIV genome from the arXiv, and synthesise that instead.

(Note: as far as I can tell, nobody's actually posted HIV to the arXiv. Small mercies.)

Re:Pause Giant AI Experiments: An Open Letter by dekhn on Wednesday March 29, 2023 @07:08PM (#35365196)
The sequence of HIV is published and has been for a very long time. In fact there's a wide range of HIV sequences: https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=...

You could synthesize that genome but it wouldn't be effective without the viral coat and protein package (unlike a viroid, which needs no coating, just the sequence!).

I should point out that in gene therapy we use HIV-1 derived sequences as transformation vectors, because they are so incredibly good at integrating with the genome. To be honest I expected work in this area would spontaneously and accidentally (or even intentionally) cause problems on the scope of COVID but (very fortunately) it never did.

One would like to be able to conclude that some virus work is inherently more safe than other virus work, but I think the data is far to ambiguous to make such a serious determination of risk.