All of those people have relationships with people already on campus, and many of them are given IDs if they're there long term. If you're just some guy with a backpack who hangs around campus for no reason and no one know what you're doing there, people are going to notice, at least where I'm from. YMMV