Re:It's Time to Investigate the AI Labs
by ben_w on Tuesday September 29, 2026 @03:15AM
(#49889365)
> We don’t even need to jump straight to legal penalties - why not just regulate that these companies can’t build partnerships with organizations in designated risky/regulated industries such as biotech, defense, etc?
This would be a good idea, but good luck, you'll need it because…
> Instead of what they are doing now, which appears to be actively courting as many of them as possible for vast sums of money.
… it's worse than that.
Anthropic may have wanted some defence sector money, but when they tried to say "no" to certain uses on the quite reasonable grounds ~"WTF are you nuts?" the US government tried to force them to supply it anyway.
And that's before we get "what do you mean by partnership?": people on Hacker News have been critical of the e.g. anti-bioweapons type guardrails on models ("censored", "lobotomised", etc.) since at least GPT-4*, so while "billion dollar mega-project headline" is obviously bad, there's an important sense in which "partnership" goes all the way down to one racial supremacist in a shed asking an LLM for help making ricin.
Here's someone three years ago replying to me to say "There’s no way to build out a wet lab with a chatbot for weapons production. It’s just not feasible. Your example is a fantasy.", and yet here we are in late 2026 with news about AI labs investing in bio wet labs and getting their LLMs to do research: https://news.ycombinator.com/item?id=38331225
Good thing it's not a weapons lab, eh? (Would anyone even tell us if they made an agent do weapons?)
* or even GPT-2, if you count all the people mocking them for trying to practice safety while failure was still not a disaster.
This would be a good idea, but good luck, you'll need it because…
> Instead of what they are doing now, which appears to be actively courting as many of them as possible for vast sums of money.
… it's worse than that.
Anthropic may have wanted some defence sector money, but when they tried to say "no" to certain uses on the quite reasonable grounds ~"WTF are you nuts?" the US government tried to force them to supply it anyway.
And that's before we get "what do you mean by partnership?": people on Hacker News have been critical of the e.g. anti-bioweapons type guardrails on models ("censored", "lobotomised", etc.) since at least GPT-4*, so while "billion dollar mega-project headline" is obviously bad, there's an important sense in which "partnership" goes all the way down to one racial supremacist in a shed asking an LLM for help making ricin.
Here's someone three years ago replying to me to say "There’s no way to build out a wet lab with a chatbot for weapons production. It’s just not feasible. Your example is a fantasy.", and yet here we are in late 2026 with news about AI labs investing in bio wet labs and getting their LLMs to do research: https://news.ycombinator.com/item?id=38331225
Good thing it's not a weapons lab, eh? (Would anyone even tell us if they made an agent do weapons?)
* or even GPT-2, if you count all the people mocking them for trying to practice safety while failure was still not a disaster.