I suspect the reason perl flourished here was a combination of luck and the cultural fit. Culture here includes the newbie-friendly online help (e.g. perlmonks), the ease of "publish and re-use components" (CPAN).
I wonder if perl would still be used if the project was started today.
Perl is however an excellent scripting language, and though it's ugly, it's just as easy as Python to pick up and use. Thus its danger.
There are other facilities of Perl though, in general its quick scripting, that make it as good a tool as any other, just so you don't get sucked into the idea that regexes are a good tool here.
I wonder if perl would still be used if the project was started today.
I think so. But would that be perl5 or perl6? :)
Because at the time Perl was probably the only capable dynamic/scripting language.