I drive a Volvo S90. I only really use the "Pilot Assist" on the highway, and once you've driven it for a while you can get a sense ahead of time what situations it will fail in.

You're right though the biggest issue is still anticipating what other drivers are going to do. You need to override the system to stay out of people's blind spots or else you'll get cut off or worse. When you stay at a steady speed other drivers seem to forget your presence quicker.