I also have this experience. I am a native English speaker who learned German quite young. I still think certain things in German, especially things that I learned either in German, or during that period of my life.

As noted, "ja, genau" still echoes in my mind for total agreement. There just doesn't seem as good an English equivalent!