Worse is better/creative limitation/blub paradox: Smalltalk, like Lisp, is too powerful; you can do essentially anything you desire in a bunch of different ways. Too much flexibility makes it harder for a clear use case to emerge and having too many choice points only makes pathfinding through the solution space more difficult unless you've made a bunch of firm decisions beforehand. Other languages like Go or Python make many of these decisions for you. It seems like the ideal pattern in these ultra-expressive languages is to construct a much more restricted DSL which can ultimately lead to a proliferation of completing solutions a la the JS ecosystem.

Inner platform/walled garden: Almost all Smalltalk systems basically implement their own operating system complete with their own integrated graphics system/window manager, process scheduler, and so on. It's an amazing environment for creating applications but it doesn't play well with the outside world and there isn't a straightforward way to 'shake' an image down to just your application. Deployment usually comes with the overhead of the entire Smalltalk system and interop with native can be somewhat limited. Imagine if Java applications only ran within Eclipse.

Zipf's law: The popular languages are so popular that their communities generate their own gravity (i.e. lots of effort tends to attract more effort). Despite Smalltalk's many appealing qualities (ergonomics, simplicity, maturity, robustness..) it stands in the shadow of giants; many people don't even know it's there or can't invest the effort to give it a shot so it never achieves critical mass.

Something that is not to underestimate is the amount of work it takes to support various kinds of content and programming languages nicely in this form. To get the most from it you want some kind of runtime integration to get feedback (errors completion) or at least good type system support integrated. It's interesting to watch a project like Atom grow – the plugin system they have is nice but ultimately the kinds of interaction they support with it are only text based. I think, in order to have a Smalltalk-like IDE take off, one needs to have similar support for extensions while allowing those to be easily integrated with richer tools like inspectors and debuggers.

I guess people are just married to their customized Vim, Emacs, Visual Studio etc. setups which they are using for many years. These Smalltalk-related IDEs are too different and break a lot of old habits.

I was myself a fairly dedicated emacs user and what I like so much about the Lively UI is that you a) get all the programmability and customizability that you have e.g. with elisp (all the stuff on the screen can be programmed, keybindings, windows, everything down to the characters in texts) while b) get a user interface that is way more powerful than simple text. It's easy to embed images and interactive objects into normal texts allowing things to be more interactive to use. It's easy to plot and visualize data (interactively, not just a static screen), e.g. see the plotly morph in the partsbin. There is a sketch morph in the PartsBin as well, allowing you to sketch and take handwritten notes, I use this in combination with Astropad. Etc.

All that stuff makes the environment much more convenient to use for general content creation, not just programming.