I used to spoof user agents, randomly mutating. Then last month I started to run into sites that refused to load/render for incorrect strings. It appears the Google and other large companies are now sending customized versions of their page based on the user agent which I can only speculate is an attempt to save bandwidth. By taking advantage of implementation specific features, they not only eliminate all of the extra JS and browser workarounds, but they also can take shortcuts.