I say that on average about once or twice a week at Google. (The other days, the firm would benefit from my further presence, and I'm there till late.) Nobody bats an eyelash.
I wonder perhaps if a large part of Google's recruitment and retention advantage is that they get this, and understand that you're responsible for your own productivity and should do whatever it takes (including taking time off) to maintain that. If you want to come in at 1 and work till 10, nobody cares as long as you're getting stuff done in those hours. There've been times where I worked 12 hour days for 2-3 weeks, followed by 6 hour days for the next week after that. The guy who integrated the PacMan doodle was in the office from about 1 PM to 8 AM a couple days before it was scheduled to go live, but then yesterday he came in at about 1 and left after dinner.
I did an internship at Google, so I have only seen one team in action for a small period of time. That said, there were a shocking number of meetings-- daily standups, one-on-one meetings, video conferences with California, weekly team meetings, cross-functionals and a few extra bits of coordination bureaucracy. These were scattered throughout the day in a way that put firm limits on how free of a schedule you can make for yourself.
Also, oftentimes interns don't have a firm grasp on which meetings they can skip and which they really should show up for. There are at least 4 weekly or bi-weekly meetings on my calendar that I rarely go to unless the particular week's topic is relevant to what I'm doing. I always go to meetings that are specifically scheduled with me, and usually go to the daily standups. It takes a while to develop the organizational awareness to make the distinction, though a usually good rule of thumb is that meetings that someone specifically sets up with you should be attended, while meetings that you're invited to simply by virtue of being a member of some group can safely be skipped.
Even worse: You'll be in the office from early to late because even though you don't love it, there's a corporate culture of "isn't this place great?" and you're worried that if you don't stay late people will discover your secret dislike for the work environment.
Never been at such a company, don't plan to ever be at such a company... but I've heard quite a few friends describe their silicon valley jobs that way.
This can clash with cultures in many other parts of the world. In many places, a job is something you do for money, you're not expected to like it, you're just expected to put up with it, go home, and use the money you earn to do the stuff you actually enjoy. Job hopping is discouraged, because it shows you're a disloyal employee and not holding up your end of the bargain. Here, the employer-employee relation is less "you do work for us and we give you money" and more "we're all in this together, and if you're not in this together with us, get your damn ass off the boat".
That was, hands down, the worst thing about my day job. Regardless of the amount of work you have to do or your personal feelings, you will be in the office from early to late because employees at companies like ours are in the office from early to late.
I don't know that "You'll be in the office from early to late because you'll love it so much you don't want to leave" is that wonderful of an improvement on this, incidentally.
My favorite part about being self-employed is that I can say something nobody can say at a big company: "The firm will not benefit from my further presence today. Well then, I'm done. See you tomorrow."