Eventually yes. At some point you need to not just solve the problem but convince me that you have solved it. If I tell you I proved p!=np you wouldn't believe me, unless I can explain how I proved it so you can replicate it. (Note if p=np it is likely I can take my proof and construct an algorithm that that can only be created if it is true - thus hiding my real proof instead showing a proof by contradiction - if it wasn't my algorithm wouldn't solve this problem that is solves)

Thus if an AI say p!=np without explaining why we won't know if we believe it unless the process can be explained.