This is so true. The starting developer will feel an immediate response, without (typically) first having to install/configure the webserver, compile code, setup deployment scripts etc.
PHP is definitely not a good language to start programming at first (for that I'd recommend Scheme, ala SICP) but it's definitely a great one for a My First Website project.
Most into tutorials start "this is an int, this is a char, this is a variable, etc." Before you even get to anything interesting you are already dozing off.
With PHP, however, you can hack together your own blog/website show it to your friends and say "I made this." The ability to say "I made this" is the best feature of any language, but usually it is a pretty inaccessible feature for the beginner.