I don't find this a very good argument. Sourcing inspiration from a sibling comment, it's also the employer's bathroom stall. I might be convinced it's okay to snoop when it comes to their network usage, but this is not the argument to do so.
The difference to me is in the purpose of the two facilities. A toilet is there for the employees' physical needs and more or less no other purpose. A computer is there primarily to do business work on. The company has a clear need to be involved in how that computer is used in several ways, such as maintaining its security, monitoring its performance, making sure it isn't misused, etc. They can afford you some privacy, but only on a best-effort basis because it's not reasonable to be entirely hands off.
As a user, you should realize that when you're on company equipment, privacy is more of a courtesy than a right. It's their equipment you're using. It's reasonable to expect them to use it in a way that furthers the company's interests. So act accordingly.
As an admin, you don't ever go digging through stuff for no reason, for curiosity, voyeurism, or for personal reasons. But again, watching out for the company's interests is part of your job, so if you run across something or have a concrete need to actively look for something (not just a fishing expedition), then lifting the veil of privacy might be the right choice or even the only right choice.
Basically, in a corporate computing environment, privacy is not guaranteed, but crossing lines should have a proper justification. In your CFO example, the sysadmin is using official powers but acting in their own interest, so that's definitely not an OK justification.