That's really not what happened at Digg. The problem at Digg was a structure which rewarded "power users" and allowed third party commercial interests to gain control of the site. It's bad enough that Digg had advertising - eventually we were looking at Digg ads to get to content that was itself advertising.
And not all user generated content is worthless. Youtube is a good example of this - I can point to user after user who has uploaded quality original content of one kind or another for years.
Yes, 90% of it is crap, but if you subscribe to the channels you like and check out the channels your subscriptions recommend, you can avoid nearly all of it. And, you know, Sturgeon's Law.
(A more real problem is that early adopters of any web community building effort are overwhelmingly just there to self-promote in one form or another.)
Regarding your second point, if you include Youtube comments, and not just the videos, it's definitely 90% crap.
Comments on youtube... yeah, they're crap for most videos. I guess technically comments are user generated content, but I never really think of them that way. Even more so on a site like youtube where the videos are the important bit.
No. The future is the next Digg, the next Reddit, etc. Just like a torrent site, when one goes down (for whatever reason), the next in line pops up to take its place (and users). The internet does not want to pay money to access a forum. The internet also does not want to be advertised to. So the cycle will continue.
Essentially for a coup of Reddit to occur you have to take a large chunk of Reddit at once. That is very hard to do.
It doesn't have a standard spoiler syntax for comments or a spoiler tag(people hack it in with nsfw tags styled to say spoiler but from the main page it will look like porn(ie be marked nsfw)).
It lacks a wysiwyg editor by default. It would be nice to have a nsfw subreddit namespace so that sex related subreddits don't take up possibly safe/useful names and so that you know that /r/humanporn isn't porn. Also a nsfl tag. Ability to expand "ALL" comments. Ability to pick percentage of home page each of your subscribed subs get.
Also a policy that allows removing bad mods from power, ie /r/Holocaust really should not be run by anti-semitic holocaust deniers.
Oh a way to sort posts by reverse age and allow posts in between time x and y not just last hour/day/month/year.
---
Of course reddit does have good mobile apps (at least on android), a competitor would need a good mobile app for at least 2 systems which can take a while.
It's having the issue a lot of 'free speech' centric services have; it appeals to a minority who feel censored, but not the mass market that simply want to discuss less controversial topics.
And until there's enough people able to talk about a variety of subjects, the site is basically pigeonholed as the 'place people banned from Reddit might go'.
Reddit, the software platform is open source: https://github.com/reddit
What about HN? I don't disagree that a lot of ugc on the web is crap. I do think though that there are viable strategies for encouraging quality content.
stackoverflow would be one counter example.
So would a lot of subreddit (IS, CS, and science oriented ones for example).
Oh and what about wikipedia?
But, more importantly, how much Reddit costs to run, at their current scale, is immaterial. Reddit has raised $50 million in VC money and is valued at $500 million, that means that half a billion is the mark they are expected to meet/exceed in lifetime revenue, independent on what their costs are. For better or worse, for-profit startups have very different incentives than non-corporate internet communities. This allows Reddit to not be 4chan (wider audience, some level of control on trolling and harassment, etc), but at the same time, it can't be 4chan (including running on a ramen-for-one-and-electricity-bill budget if need to be)
Normally I'd suggest they start another business doing some honest work (i.e. not ad revenue). Say, a hosting service, since they have experience with that already. Reddit then could become a marketing expense for them - something they'd run for the sole ability to say "Yes, we're the ones who're running the Front Page of the Internet! As you can see, we have some experience!".
But, as you said, they took VC money. It changes the picture significantly. I'm not sure they can be helped.
In every get-rich-quick scheme there's a catch. In startups, the catch is that if you care about your product then taking VC money is a difficult decision because it means surrendering your business to the whims of people who absolutely don't care about your product.
Craigslist wiped the entire classified ad industry and much of the newspaper industry off the face of the earth merely by being the low-cost provider. Ad-supported businesses should be afraid. Very afraid.
As for Reddit, the VCs may have to take a haircut.
Is the content mostly text or does it have a lot of images/videos/sound files? If the former, it's a lot cheaper to run than the latter (hence why a lot of big community sites are so 'vanilla').
What's the audience/topic? Some fields are easier to monetise than others. People in tech are more likely to use Adblockers, but they're also more likely to try out new methods of monetisation (like say, a Patreon type system or something involving Bitcoin).
How much do you know about running a server? Much cheaper to run a big site on an unmanaged service than a managed one.
How many people post vs how many people read. More posting in less time = more server resources and more service costs.
What script do you use? Something lightweight and lacking in a lot of features is going to hold up for longer on a shared server than something like XenForo or IPB or VBulletin.
So it can be really cheap if you're after a simple forum with an audience that doesn't block ads and might be willing to spend a lot on subscriptions or donations. It can be really expensive if you're offering a live video chat microblogging/livestreaming service.
I've seen large forums with hundreds of thousands of posts on $5 a month hosting, and sites with about 50 users needing a dedicated server.
As long as content is hosted on company-controlled servers where ad revenue is the model for survival, you're going to see slow declines and eventual busts like Digg. Users hate being advertised to and censored.
The future is probably something where people pay a small amount to access a forum that's somehow distributed on their machines (phones, PCs, etc) and not controlled by any organization. Data transfer (even mobile) and storage are getting cheaper. It'll be like bitcoin, but for discussions.