With ARKit you could probably get it to recognise and track the controller using it's object recognition, there's sample code here : https://developer.apple.com/documentation/arkit/arkit_in_ios... Then you wouldn't have to strap the phone to the controller

I'm not so sure about RealityKit, I found it doesn't have the flexibility and finer grained options of ARKit, it's annoying that Apple make 2 frameworks for AR but don't have the same features in both