I'm actually curious what systems you're using. I have very limited experience, I have a M3 and a 21 Corolla with Toyota's version.
My M3 has never once just given up on the lanes and stopped steering, but it's a very common occurrence on the Toyota. It just gives up and suddenly you're back to controlling it with no real rhyme or reason to it that I can find.
I guess it comes down to preference. I see it as assistance. I want a helper who maybe can’t do it all, but never fucks up.
I've used Blue Cruise a good bit which supports hands-off modes. This system works very well on highways hands-free. It seems to want me to put my hands back on the wheel when approaching the toll gantries and goes back to hands-free a few seconds after the toll gantry, but otherwise I've been able to go well over 100 miles at a time without touching the wheel or brakes. I've never had any kind of phantom braking situation or slow downs, it works well in stop and go traffic, and has overall been a great experience.