This might seem reasonable, however, this is only a false economy as people tend to gravitate towards common words that evoke simple concepts, which drastically reduces the search space.
A dictionary attack isn't anything novel and the proliferation of this technique would only make life easier for the baddies.
You could argue that passphrases aren't even competing with randomly generated passwords but rather with the naive passwords that people come up with that has simple character substitution for special characters.
However, this is the same reason why it should only be confined to use in places where a yubikey for active directory windows login or a password manager wouldn't be practical.
If you're going to use a tool to generate your passphrase, why not use a tool to store it and retrieve it at will, while also protecting yourself from shoulder surfing?
This is a case where I choose to believe xkcd unless and until you can provide convincing evidence otherwise.