While they're not building AGI, I would argue that their thinking constitutes not only work, but progress. MIRI is attempting to flesh out the murky, abstract nature of the control problem and translate it into something more concrete, specifically mathematical models that may one day evolve into provably-sound foundations for future work.
>Once it does, the field (in theory) doesn't really exist anymore really. So there can only be a research into it before it is complete.
It is possible and perhaps even likely that the first AGI we create will be contained, but otherwise unsatisfactory to release or utilize without further iteration and study of the design. After all, it's probably a bad idea (in most cases) to leverage its superhuman intelligence if you believe the thing is malicious or otherwise has motives that do not align with human interests.
That said, I cannot imagine research proceeding in any sort of public fashion once that threshold has been passed, so in a way you're likely correct.
>Arguing about safety is totally a worthless effort, and harmful in my opinion, when we have no idea what the foundations are or the paths to it.
While I agree regulation is premature, prominent people like Musk advocating for AGI safety, if only in a general sense, strikes me as a positive thing due to the cultural effects.
Prior to Musk, Hawking, et al., if you mentioned the prospect of unsafe AI, you'd be laughed at in all but the most niche communities. The notion was clearly just something confined to the realm of science fiction movies. Nothing with similarly catastrophic ramifications could ever happen in real life, or otherwise such things were indefinitely far off in the future.
I'm not advocating fear mongering, just saying that when AGI actually does become a serious safety concern, it's better to have the cultural climate already primed and receptive to addressing the problem. On the other hand, if the safety angle is pushed too hard now, it may backfire and have the opposite effect later, and that would be unfortunate.
>There are no rules stating that a project has to be turnkey to realize a goal.
No, but in practice there are political and financial constraints that can sometimes have a similar effect. If a project isn't properly aligned with those constraints, then it becomes more likely to fail. The longer and more expensive a project is, the harder it is to justify or continue to justify in such climates.
No, like Joscha Bach, Yoshua Bengio, Ben Goertzel etc... the MIRI people are thinking about AI, not working on it. I should have made that point clearer and said as much. For AGI building = research, thinking =/= research. So for example, my Masters thesis is AGI implications of a nation state building it. This includes several rough roadmaps to AGI. I would not call myself anything close to an expert or researcher working on AGI because I am not building systems that are testing hypotheses about machine intelligence.
AGI does not yet exist.
Once it does, the field (in theory) doesn't really exist anymore really. So there can only be a research into it before it is complete. Arguably an AGI's (if you fall on the side of it being complete at "birth") recursive self improvement makes human input basically worthless.
The only consensus we will come to is when we all agree that the thing we built is actually an AGI. I can talk about those measures but at the end of the day most of the leading people think it's going to be "we'll know it when we see it."
Unlike atomic weapons prior to the first test shot (or even prior to Fermi's Chicago Pile-1 experiment), AGI does not yet have a solid theoretical underpinning, let alone a proof of concept.
Exactly. Which has been my point all along. Arguing about safety is totally a worthless effort, and harmful in my opinion, when we have no idea what the foundations are or the paths to it. When people are calling for regulation before we have anything like real funding for it it totally stifles the field. At best it diverts resources to making "provably safe" AI which is an endless rabbit hole in my opinion because it's an impossible metric.
there may not have been enough time for the aforementioned theoretical underpinnings to have been reasoned about to the level they were.
Enough time? That makes no sense. Lets instead call it an "Delta project" for AI - the entire front end of the project can be building the theoretical framework. There are no rules stating that a project has to be turnkey to realize a goal.