Fair enough. But I'd assume that there's an implicit understanding in that agreement that if you're needed for something work-related you probably shouldn't go to yoga, i.e. they assume you'll be an adult about it.

Correct. It's pretty loose and varies from team to team, but the generally-applicable rule is "We have OKRs each quarter. If we're not hitting them, that's a problem." How a team goes about hitting them is left as the responsibility of the team.

One way that process can break down is if you have a manager that's bad at planning or over-promising for your team without testing the wind of how much work their engineers will actually get done. Google makes it pretty easy for demonstrated high-performers to change teams, so if you do that too much you run the risk of hemorrhaging good team members (who can't get promoted if goals aren't met and therefore will seek out a team that knows how to meet goals) and being left trying to get your tasks accomplished with only the people who, for whatever reason, can't transfer.