>but of course that's not what we'd do. More likely we'd burn the designs, lock the inventor up and throw away the key.

Which is exactly what we should do (in fact, the inventor shouldn't be locked up, he should be "disappeared"), if we want to avoid horrible destruction.

I really think this kind of thing might be the answer to the Fermi Paradox: Just imagine if any idiot could easily construct not just any bomb, but a doomsday bomb that destroys the whole planet. The planet wouldn't last long, because sooner or later some mentally ill person is going to build one and set it off.

You could deal with them that way, sure. But every other person after them will learn not to divulge it and take the best offer from other countries (submission mentions offers from France and Pakistan for Phillips) and give it to them, because you're not going to reach a worldwide consensus on "let's get rid of every person that develops X theoretical doomsday device" and others will want that edge. This seems so shortsighted it'd do well together with the vitrum flexile story. Just like engineers after WWII, during and after the Cold War, you'd want to give them a comfortable position using their talent for something more productive than developing military-purposes missiles. I'd be more worried about biochemical warfare anyway, I'm surprised there hasn't been any major attack with water supply as a vector.

>This seems so shortsighted it'd do well together with the vitrum flexile story. Just like engineers after WWII, during and after the Cold War, you'd want to give them a comfortable position using their talent for something more productive than developing military-purposes missiles.

This isn't like that situation at all. The WWII scientists could build bombs, but not easily: it isn't easy to get your hands on enriched uranium or plutonium at all. It seems like very few people can understand me here, so let me try to re-iterate: I'm proposing a situation where anyone able of operating a smartphone is able to download the information to build a planet-busting doomsday bomb, and build it in his living room using readily-available, common, everyday tools and materials. Once the knowledge is out there (thanks to the internet, which also didn't exist back then), you can't really squelch it. And you can't restrict the tools or materials needed because they're common. The only thing that was missing was the knowledge of physics needed to put them together.

For a somewhat similar story, look at Harry Turtledove's sci-fi story "The Road Not Taken", where some slightly dim-witted aliens attack Earth using medieval cannon-type technology, and hilariously lose, but were able to get here because they discovered some overlooked but simple branch of physics we somehow missed, which allowed them to easily achieve FTL transit.

https://en.wikipedia.org/wiki/The_Road_Not_Taken_(short_stor...

Why bother with water supply when you can use the air flow :(

You could just offer (with the alternatives maybe making it a "can't refuse") to put the inventor to work doing something more benign and pay them for their skills. You could even let them do side projects like these at the lab so they get to scratch that itch.

Not everyone needs to be disappeared.

What happens when the information gets onto the internet and proliferates though? Then it's too late and humanity is screwed.

If we carefully think about it, someone is going to do it judging from the fact that the person did it purely from textbook based information. I’d rather think it’s better to identify ways to build such things and prevent wide scale adoption in a reasonable way. Locking up people and burying the evidence is a short term solution. Building ways of resilience and to be able to thwart or diffuse such bombs or to make laws which make it harder to extract raw materials seems a good long term solution to think about.

>I’d rather think it’s better to identify ways to build such things and prevent wide scale adoption in a reasonable way.

But how do you do that? Suppose some new physics are discovered, and this makes it trivially easy to build a doomsday bomb using readily-available materials that simply can't be restricted.

>Locking up people and burying the evidence is a short term solution.

Unfortunately, yes: if it gets to my scenario above, it's probably just a matter of time.

>Building ways of resilience and to be able to thwart or diffuse such bombs or to make laws which make it harder to extract raw materials seems a good long term solution to think about.

Again, in my scenario, anyone with half a brain can easily build the doomsday bomb using stuff from the grocery store. We're not talking about plutonium here, we're talking about very common materials. How do you stop the world from being destroyed?

I think I also remember that interview or maybe a YouTube video about it. Something like what would we do if someone discovered that you can put sand and magnets into a microwave with the plate rotating counterclockwise and it goes gigantic boom. Have a look at the Veritasium YouTube channel, I have very vague memories that it might have been there. Or maybe Vsouce or Kurzgesagt.

EDIT: The Vulnerable World Hypothesis [1] with the Easy Nukes thought experiment. And this a TED talk with the author [2] and probably what I watched.

[1] https://nickbostrom.com/papers/vulnerable.pdf

[2] https://m.youtube.com/watch?v=JrjjOGI6YB4

I am afraid that current AI models are the black balls. For example it is now close to impossible to verify anything, even that whether the Russians nuked half of the country or not. (It is somehow doable with trusted chains, but can you really trust them?)

Also I suspect that hacking AI will be much more powerful than any hacking with hand soon.

I wouldn't argue the current models are, but we are definitely up to our elbow in cookie jar at the moment.

This doesn't seem to be it I'm afraid, but thanks for trying, and that's an interesting contribution to the discussion in itself.

> More likely we'd burn the designs, lock the inventor up and throw away the key.

Which is depressing since nuclear armed nations dont get invaded or subjugated. Arguably the inventor should be given the noble peace prize every year going forwards.

Well, some sort of Nobel prize may have been mentioned in the awards reasoning, but the premise in this case is more in the spirit of '640 kilotons ought to be enough for anybody'...

Nuclear armed private citizens may not get invaded or subjugated either, yet it's hard to see that the home atomics revolution would make us collectively safer on the whole.