I had the same experience on Android Wear -- interacting with apps is painful. Further, apps on the watch tend to run the battery down faster. So I wound up removing all the 3rd party apps that I'd downloaded anyway.
I discovered that if all you want is activity tracking, notifications and calendar, other brands of watch do this for far less money without the hassle of charging every day. Check out some of the newer Garmin or Fitbit watches.
I think that changes a bit with the series 3 and LTE support. I would be willing to deal with cumbersome interfaces more frequently if it meant I could leave my phone at home and still have access to essential functions (phone calls, text messages, etc). And I could see myself downloading a lot more 3rd party apps to have with me just in case (slack, 1password, authy, etc).
The only thing stopping me from getting the series 3 is the fact that I get cell service through a cheap MVNO that doesn't support the apple watch yet. Hopefully it's just a matter of time now before it is possible to use apple watch LTE service with MVNOs.