I find it akin to how if you know a photo's URL you can view it regardless of rights. That makes perfect sense if you know how web servers work and that there is enough randomness in the URL that it really is secure, but to a lot of people it feels wrong. I know 90% of my friends would be shocked that I could post a public link to their beach photos.