Reddit is successful, in that it's valued by many. It was successful for the founders [I'd dearly like to know how much they got, to an order of magnitude]. It's only 5 years old - it took Amazon 10 years to become profitable, google began in 1996 was incorporated in 1998, but didn't even start serving ads til 2000 - which accounts for almost all revenue today. So I think it's too early to call reddit "not very successful". [It would be interesting to have data on how long it typically takes for "successful" startups to become successful.]
I do disagree with pg that the essence of reddit was "to tell people what was new". Slashdot already did that and other news sites. Reddit's (and Digg's) innovation was that everyone could vote on comments. [But pg has far better information on reddit than me, so I'm left wondering why he thinks that...]
YCombinator does seem more successful than you mention though: they've turned out a lot of companies that, while not always huge, are making money. A few of them are even game-changers in their industries, like Scrib, Posterous, Justin.tv, Dropbox.
I observed this in 2007FEB22 , you can see it here ~ http://www.flickr.com/photos/bootload/398269769/ and the classic response ... "... wouldn't have been a timeout; we probably restarted the server ..."
Though I wouldn't call a broken piece of a webapp (hackernews) a failure of Arc.
Yet... HN grows and grows... it is reported widely... luminaries routinely post here... and here are you and I.
pg has said he is careful to focus on what matters (eg. spam; ranking), not what seems to. It appears to be working. And it exactly supports this essay's thesis, of working on looking professional vs. what's important. Maybe it's a bit like undergraduates handing in essays in a fancy folder?
Were there no mp3 players b4 Ipod. Was there no Lotus 1-2-3 b4 excel spreedsheet. Were there no other search engines prior to google. So really providing the best tool to solve the problem is not enough to guarantee success.
So how do you define success? Is Arc successful if you open a web page and then 30 minutes later try to submit a form and get "unknown link" or something to that effect. It seems massively broken to me.
I'm not trying to pick on pg. I'm trying to present the alternative which is this: survivor bias. pg may have been in the right place at the right time with viaweb. As an incubator, pg probably made money on reddit, but conde nast has almost certainly lost money on reddit, so I'm on the fence about that.
Continuing on, I think it isn't unsurprising that putting $15k or so into 118 startups over the past 5 years would lead to one or two successes. pg has the benefit of being the magnet, or is it magnate? All the best ideas come to him, but there have only been a small handful of "massive successes" turned out. With his social network and contacts, quite frankly, I expect more big successes than I've seen.
How successful is pg, really? Do we know?
I suppose, really, my point is this: If there was a science to this, then it should be repeatable. Since it doesn't appear that pg has repeated his viaweb success in over 10 years, I wonder where is the science? Is pg just someone we choose to listen to because it feels good? He makes hackers feel good and important and understood? He can empathize with us, so we lend him our ear?
But are we wasting our time?
I love hn, I love listening to pg on stage talk about startups. It's like chicken soup for the startup soul. But maybe I'm being misled. Maybe it's just feel good, irrational stuff that isn't practical or beneficial -- and perhaps even harmful.
How do we know we won't be listening to him and then 10 years from now, we find out he's lost all his viaweb money. He hasn't been able to keep the startup chuck wagon turning out profitable businesses or something like that and we wasted all this time trying to replicate his success because... well... he writes a lot and it feels good to read it...