I'm not sure about Google, but I know that many companies segregate contractors into other buildings. At one of my previous employers pretty much everyone I interacted with was a full time employee as well, but that was only because contractors largely did not have badge access to the office and worked somewhere else.