Apple is willing to pay the cost to minimize the margin-of-error at every step.

Another approach to keeping yield high (at the possible risk of hurting your brand) is to do what Intel did with their Celeron line. The equivalent for Apple would be a "value iPhone" which would not come with the same set of expectations from customers.

It is also easy to make thousands of identical items when your build material is solid and can’t be stretched

As for the Xbox controllers, it could be a lot of differences. Microsoft does not feel the need to keep different runs exactly the same. They may be more willing to buy different types of plastic depending on what's cheapest this quarter.

Or it could be much simpler, maybe iPhones do feel different. How much time do you actually spend interacting with someone else's phone (Have you ever even actually touched another person's phone, or just their case?), vs how much time did you spend with your buddy's Xbox controllers?

Hard materials are easier than soft materials in some ways. If you lay a soft fabric out before cutting, is it slightly folded, perfectly flat, taut? It will all make a difference.

Moulding plastic or cutting a hard material like aluminium is more amenable to tight tolerances since you can use the same mould over and over, use the same cutter and the material doesn't flex, stretch or change in any significant way.

It probably wouldn't be if human beings had to drill holes in the PCB by hand, place and solder components by hand, and hand-machine the case.

different from my friend’s Xbox controllers even though they were the same model and supposedly manufactured the same.

This could be things like slight changes in the sourcing of, say, springs for an analog stick, composition of rubber used under other buttons, and actual wear on components of the assembly line, like the molds used for injection molding of something with a pebble texture. Not to mention, obviously, that each controller will break down differently over time depending on use.