People are considerate enough not to bring poorly-behaved dogs to work. I've run into many dogs around the office and have never had a barking problem - they usually just roam in a small area around their owner/team, quietly.
Pretty much the only time I've heard a dog bark at work was when someone accidentally kicked it...
That sounds quite distracting in a small/medium office space by itself.
My experience has been people who bring dogs to work are often oblivious to the nuisance they cause to others who try to focus.
Besides, I don't work at Google. At the companies I did work, when someone brought a dog to work (few times) it was always a problematic situation, people tried to ignore the ruckus, moved to nearby desks etc.
I like the dog policy, because every so often I have the nice surprise of being able to pet a fluffy dog, at work. In practice it's not so bad as what you're thinking it might devolve into.
And how does that work for people trying to concentrate while working, with barking etc? Or are those dogs parked outside of the office?