I had the same feeling as you. I became ok with it after I realized it could be removed in the future.

http://daemon.co.za/2014/03/why-wrong-to-be-afraid-angular

I am looking forward to trying react though, because of how well it plays with backbone (which is still my preference)