But if your chip heavily builds on arm's IP you need a license for that at least as long as you can't replace the infringing parts of the design. Which sounds very much impossible if you also want to progress on other aspects of having the best chips.

Apple uses the ARM ISA. It doesn't use ARM IP - as in, the design of the chip. Apple designed their own damn chip!

Since they're not branding it as ARM in any way, shape, or form, and they have a perpetual architectural license to the ISA, I suspect they could do pretty much what they please - as long as they don't call it ARM. Which they don't.

During the design were they careful not to create any derivative works of arm IP and/or not to infringe on any of arm's patents?

Apple co-founded the company. I imagine that they have a right to that IP.

Even if not, then yes, I would imagine that a behemoth of IP design like Apple would probably have considered IP rights during that design...

They definitely call it ARM in developer tools and technical documents.

I doubt that will be true going forward. They first used the “Apple Silicon” branding for the Mac and yesterday they used the branding for the Watch and iPad (?)