I think so, too. I imagine it's the same with opting-out of "Location History" - you opt-out of seeing the log, not out of locations actually being logged.

It would be cool if there were a way for Google to prove (maybe cryptographically?) the absence of some data on their servers, so that I could verify that my location data has actually been deleted.