> If mod_perl was too powerful and had access to too many of Apache's internals, why not just create a lighter version or make it configurable?

The perl people didn't want to? My recollection of mod_perl was that you really needed to have everything about the server configured around the perl system, and it was quite a pain to get configured and working properly. My sysadmin skills weren't all that great 20+ years ago, but I did get it eventually working, but anything you wrote would only be able to be run on another mod_perl system, which were relatively uncommon (compared to the shared perl/cgi setups).

PHP was really more powerful and easier to read for many common scenarios compared to perl in those early days.

And while there were some scenarios where mod_perl was indeed more powerful... my experience was 1) those scenarios were relatively rare and 2) the community of people who had that experience was small, and never grew much (even when perl usage was on an uptake).

The problem with mod_perl for hosting companies was that it was much harder to protect global variables from being visible from other accounts on the same machine. It also consumed a lot more memory than mod_php from what I remember so it never became a part of the shared hosting scene which enabled PHP to flourish. Mod_perl was also a lot more complicated than CGI Perl which is why Perl remained only a CGI option on cheap hosts. Some very big sites ran on mod_perl but they were running on dedicated hosts, not shared.