The comments on this are interesting. There are a lot of assumptions from different people speaking about humanity as a single entity that decides how to operate as one. A lot of others act as though it’s really easy to get world governments to all agree to not develop a technology through treaties and agreements. One person below is arguing with me by saying that we “just” need to get China and the US to agree to halt AI and the problem is solved for the whole world.

It’s difficult to understand where this all is coming from because it’s so strangely disconnected from history. It’s not even consistent with current world events. The US is in a war with Iran over nuclear proliferation while also worrying about North Korea’s nuclear program. China and the US are in a trade war and Taiwan is under threat due to semiconductor capabilities (among other things, obviously). Open weight models allow anyone in any country to do AI work on their laptop. The top story on HN yesterday was a small team of academics making some cool discoveries on improving LLMs without frontier lab level compute.

How anyone can see all of this, in addition to decades of history of attempted nuclear diplomacy and wars, and posit that the whole problem is easily solved by simple agreements is confusing to me. The confidence with which these alternatives are being proposed and the unwillingness to acknowledge second order effects (like AI researchers moving to a different country and continuing to work) is baffling.