That doesn’t mean it’s allowed, only that they haven’t blocked it again. You can’t use your neighbors’ pool without permission just because they don’t have a guard checking IDs, either.

Turns out, I can.

Up until they notice and decide to call the police.

I use an adblocker, call the police and tell them I told you so.

If you found people who make a habit of swimming in your pool without permission then I imagine you'd be the one calling the police.

Not legally, no. This thread is the lower stakes version of what happens when the owner gets tired of free loaders and starts enforcing their legal rights.