And for any language that's popular now, there was a time that it wasn't yet popular, and deployment was a pain.
So I think you have it backwards: Lisp is not unpopular because it's hard to deploy; it's hard to deploy because it's never cracked the popularity threshold.
Lisp is notoriously hard to deploy. Code is not portable between different lisp implementations. Apple's ATG ended up axing many projects because they had so much trouble moving their lisp from 680x0 to PowerPC.
The fact you talk about "your system's package manager" ignores my point -- I'm not saying Lisp is especially hard to deploy to servers (although it is -- but this is, as you say, a popularity issue; and it would be far worse if the world of servers hadn't converged on a small number of very similar platforms). I'm saying it's hard to deploy to desktops and mobile, which is where most of the "useful" programs people might want to write still live.
A huge part of Lisp's problem is fragmentation. Which Lisp? Aside from fragmentation in Common Lisp itself, there's all the not-quite-Lisps around, ranging from Arc to Scheme to Clojure to sheesh-I-really-have-no-idea. And of course this is because once you start coding in Lisp there's an overwhelming urge to abstract away difficult things in your own clever way, perhaps even hacking the language itself, and pretty soon your little world is utterly unique.
Another great, simple language in the exact same boat is Forth.
Why is this? It's because they can live in, and interact with, the common linking and loading ecosystem that the rest of the world lives in: the C ABI. If you can link your output into something that respects the C ABI, you can deploy it[1].
Common Lisp implementations could immediately solve their deployment problem by living in this ABI. So far, the only one that has done so is ECL (which is an excellent project).
[1] A few alternatives exist. The Java and .NET ecosystems are also viable link targets (as I think Clojure has proved).
Lisp systems like LispWorks can generate typical platform shared libraries.
Many can easy embed C-code.
Some compile to C, which than can be embedded. ECL is only one example. Others are mocl or the ECL-related systems.
But this only restates the point. It's easy because those things are popular. Try deploying an executable that requires a shared library that you can't download with your package manager, and you're pretty much back into the same pain.
Using the C ABI helps if you want to link against other code that uses it, though it isn't strictly necessary (pretty much all these language implementations have foreign call mechanisms). But this is a development issue, not a deployment issue.
ECL does make it convenient to build standalone executables, but most CL implementations can also do this.
The idea that you can write a useful program in Lisp faster than in other languages is pretty unsupportable. For most users, the time and effort spent getting Lisp up-and-running on their machine of choice will already have lost. Incidentally, I played with the interactive REPL and it wouldn't advance beyond the second page.
And, if you do write a useful program in Lisp, you run into the deployment problem. The deployment problem is pretty bad. Apple's Advanced Technology Group at one point had one of the highest concentrations of Lisp-heads in the world and produced a bunch of stuff like Dylan, Sk8, etc. that was intimately tied to their Lisp implementation. Not only was this not cross-platform, and pretty hard to distribute and deploy even on the platform it was created for (680x0 Macs) it was killed by the transition to PowerPC.
This isn't as big a problem if you're writing server software, but it's absolutely disastrous if you're writing desktop or mobile software.
One could show someone how to do something useful and deployable in Python, PHP, Perl, Javascript, or whatever in a few minutes and chances are they could have a usable dev environment set up in a few minutes.