A Friend of mine and I had attempted something similar but in an indoor setting. The key problem we faced is that the compass will in the phone and positioning system. The way we overcame the orientation was by using the camera to combine the visual odometer data with the compass measurements. We also faced issues with the localisation as being indoor meant we had to rely on wifi location which was not very good. Again, the way we got over this was to use the visual odometry, and rely on users turning. https://github.com/chelseyong/Android-App-AR-Map-Navigation