It comes from the threat model, having a password book on your desk in a cubicle is absolutely not secure.

On a desk at home? It is marginal, certainly a burglary is a low frequency event, but we also have events like fire that make it insecure in other ways.