Yes, you could. That would be easier to remember, but also easier to guess. If your phrase is long enough, and unique enough, that wouldn't matter. The mere threat that it could be any character is enough to discourage most attackers.

Probably the best argument against spaces is the attack that listens to the sounds of your keyboard with a microphone as you type. As the space bar is a larger key, it sounds a distinctively lower note as you type, and would give even an unsophisticated attacker the means to determine the word lengths in your passphrase, which might reduce its entropy to something guessable within the lifespan of the universe.

Probably not a concern unless you might be targeted by someone with government-level resources.