I think three things contributed to its downfall:
1) Perl 6
2) the huge inflow of people into Unix who were not keen on Perl's ultra-Unixy wackiness (let's borrow from bash, awk, regexps and everything under the sun! - fun but not exactly welcoming to require newcomers to learn the entire cultural history of Unix to learn your language)
3) the fact that sysadmin work was quickly graduating from writing scripts that send angry emails when users fill their 20MB home directory quota into managing serious, distributed system that serve as crucial infrastructure, for which the non-wackiness of Python was a better cultural fit (particularly when the role of developer and sysadmin started to merge)
It's not so much that people started rewriting their Perl scripts, it's more like the newcomers who weren't doing Perl soon outnumbered those who were.
Perhaps one other factor is the rise of GNU/Linux and death of Unix. Linux might still be the world's most "self-incompatible" operating system -- but it's a lot more coherent than a mix of various flavours of Unix/Solaris/BSD etc.
Shell scripting makes more sense if you can assume a certain lower common denominator of GNU sed/grep/awk and posix and/or bash. Along with more or less consistent file system layout, location/availability of special files etc.
Add to that the rise of XML, then json, and perls built in regular expressions doesn't set it so far apart any more -- you'll need a library/tool for parsing/serializing anyway.
Still think it a bit odd to call it the "vacuum left by perl" -- it's probably more the flood created by python/ruby/newcomers...
Python is probably an exception there because it got popular despite the lack of a clear reason (I would argue it was there to fill the vacuum left by Perl). But Rails is what brought Ruby into the mainstream, and C was brought to the mainstream by Unix, and JavaScript was brought to the mainstream by Netscape, and so on.
Lisp never became mainstream because nobody ever wrote a killer app/library/framework/OS in that language. The most popular flavor of Lisp today, Clojure, managed to carve a niche for itself because its community is so practical-minded and has been working on very practical stuff like Om, Storm, Reagent, etc. Hopefully they will find their own killer app one day.