More generally, we're definitely committed to making SmartOS the best environment for the modern cloud; I'm sorry that your experience was frustrating, and if you have any additional concrete feedback on what you'd like to see improved, we're all ears. And finally, I know you said that you "don't care" about DTrace, but I just can't help my (biased) self; if you're developing a node.js app, you really should check out some of the things that SmartOS can uniquely do with respect to node:
http://dtrace.org/blogs/dap/2012/04/25/profiling-node-js/
http://dtrace.org/blogs/bmc/2012/05/05/debugging-node-js-mem...
Sure, I didn't spend a lot of time with it, why should I? I saw that it was missing a great deal of basic things out of the box. I decided quickly to not waste my time learning more esoteric stuff that is only useful if I stick with Joyent and SmartOS.
You are an absolute genius for creating dtrace and it is nice to have but certainly isn't a necessity for most applications. Your example is voxer which is a total edge case since they've been pushing the node envelope since day one.
In the end, the reality is that SmartOS doesn't feel modern at all, it feels like Windows98. Actually, it feels like SunOS back when I had pizza boxes under my desk 15 years ago. I could spend time giving you tons of (free) feedback about what's wrong with SmartOS or I could just use Ubuntu and move on.
9900+ concurrent connections on a $22/mo Ubuntu box is 'good enough' for me.
A few days ago there was an article about Linux monoculture. It's even worse than that, it's an Ubuntu monoculture.
If you really believe there is more information in SmartOS' wiki than in man pages & other proper documentation available about Linux, I don't think the problem is with the Linux monoculture, but somewhere else.
Joyent has to make the experience better by streamlining documentation, not by compromising its core values that make SmartOS great. The quality of the Solaris documentation is great, it's just not organized how Linux people expect it to be and they don't know where to find it.
The bigger issue is Joyent's product lifespans. We've had to migrate our servers twice in that three or so years. Because of that, I moved to a different cloud provider. I don't know if I can expect better.
The only outage I can recall was recently when they began the process of EOLing the current product we use from them.
As for your bash script... I stopped compiling binaries ages ago, why would I want to start again when I can just use Ubuntu and the official NodeJS PPA?
I don't care if SmartOS is 'faster' or has dtrace. The fact that it is missing basic things like top (no prstat isn't anything close to top) and 'standard' things like upstart made me want to run for the hills. The Joyent wiki documentation on SmartOS is abysmal at best. Even the pkgsrc repository system feels ancient and pales in comparison to what apt is. I have no desire to spend time re-inventing basic tools that linux has had for ages.
The image version of SmartOS w/ NodeJS isn't up-to-date with the latest Node release and it starts up MongoDB by default. While this may be good for just getting things running quickly, what happens if I don't want Mongo running at all? Generate my own images? No thanks.
I know, lots of hate... but it isn't all bad. Joyent as a cloud platform is great. Basic ab testing against NodeJS had me serving 9900 concurrent connections without any issues on their least expensive vm (which runs on a blazing fast network at 10gb/s). We were on Rackspace before and with a network limited to 20Mbps/s (at the lowest tier) we just couldn't get the throughput we needed.
tl;dr: SmartOS is a train wreck. Don't waste your time. Joyent as a platform is great. Why bother with EC2 when Joyent is faster and less expensive?