Look, I have two objects in a locked room. They are in fact very similar objects. They have the same weight, same visual presentations, same physical composition, up to reasonable limits of measurement. In short they are very similar objects. Your approach says, the object is only similar iff someone has the key to the room and observes the two objects. Sure, the only way to tell that they are similar is to for someone or something to make the observation. But its quite something different to say that they are similar only because someone observed the two objects. Maybe that's true, but that's a pretty wild claim.