Don't worry man, I've got you covered. Key revelation coming up.
I suspect you're thinking of this in the context of getting a single PHP app running on your box. What if you have several? Or what if you want to have several branches of the same codebase checked out and running? It's doable with some VirtualHost kung fu, maybe a wildcard ServerAlias, but it's no longer as simple as 'apt-get apache'.
Much simpler to tell someone, "Do a 'php -S' in whatever project you're working on at the moment."
This is EXACTLY why it would be awesome to have a web server bundled with PHP.
Your scenario of running a simple test.php from the default DocumentRoot is a very contrived one, one that only the very first beginner will ever use, and even then only because he doesn't know how to configure Apache. Plus it assumes a lot of things: that it's on Linux, that it uses the distro-supplied packages, that they use a setup that is the very basic one, ... OK for beginners, but serious friction ('friction', not 'unsolvable problem' - that still doesn't make it unworthy of being addressed) for many experimentation scenarios.
Also running php -S in a shell is simple, run it, and minimise the window.
Your statements are only true for the simplest of cases.
I've been working with both LAMP and WAMP (Windows version) since 2003 - about 7 or 8 years now...
And even provide a pre-packaged WAMP type-distribution with a Pro edition and a Community Edition: http://www.devside.net/server/webdeveloper
I'll skip you the drama, but take my word for it, getting AMP to work well together (and working with AMP) is not a trivial task and requires literally years of experience. There are about 1000 ways you can screw it up. You might laugh at this statement with your "apt-get" or your Windows-based Apache setup.exe, but what happens afterwards? And what happens if there is a problem?
Having provided support for so many years, I can tell you it can be a nightmare for some people.
In fact that's part of the reason why PHP became so popular: 1) it's free, 2) it's easy to do basic html manipulation, and 3) it's easy to get a basic setup running.
Why bundle a custom web server with a scripting language when literally all you have to do is visit one more web site and download and run one more installer? I feel like I'm missing some huge key revelation here because I keep getting downvoted for these thoughts. Should PHP also bundle their own custom MySQL engine so I don't have to run 'sudo apt-get install mysql'? Or a copy of Tuxracer too so that I can play it when I need a coding break without having to run the installer?
Edit: in fact, at the risk of further downvotes, I would go so far as to say that if it really is that difficult for someone to do a basic Apache/mod_php install in their OS of choice, they should probably do more reading/tutorials before proceeding down the PHP rabbit hole any further. Figuring out how to install a dev instance of Apache isn't more difficult than reading the documentation for getting this new embedded PHP server running.