When we say "X is beautiful", it is actually shorthand for "Lots of people find X beautiful", or "The consensus is that X is found to be beautiful".
/aspie
Not everyone would agree to this. There are people who believe that objective beauty really is "out there" in the world, independent of human judgement.
Your opinion is very fashionable at the moment, particularly among academics and intellectuals, and similar arguments are applied to morality and anything else not subject to empirical investigation. But as I said, this way of thinking is by no means universal.
If you believe in god, space aliens, etc., we can shift the admirer of beauty to another perceiver, but still, without the perceiver there is no concept of beauty ascribed to the object.
People are weirdly insecure about their taste in visual art. Nobody worries "Uh-oh, what if I like music that's actually bad?" I guess at least with bad music, you feel like you have company. If you're into Eminem you'll meet other people with Eminem T-shirts, but you don't see people with Thomas Kinkade or Jeff Koons T-shirts.
You know if you still like it, but you don't know if it is beautiful. You might like ugly things.