I agree, I don't see what it adds. All it means is that the AI doesn't know what it's doing -- but it's still doing it! So as we're getting converted into computronium, I guess we can take solace in how the AI doesn't know that's what the numbers mean?

With that said, it is a great proof of concept for something in the Chinese room debate: "See, this computer knows it's running deep net for someone's calculations, but doesn't know it learned how to have a conversation with someone and was carrying out said conversation."