> I agree it may be possible to develop an attack against the secure enclave with a big enough budget

You are assuming it's perfect in conception, design and implementation. That is a very unlikely assumption for any system.