As a frequent reader and reviewer of CS papers I totally disagree. Peer review helps narrow a huge incoming flood of new research into a smaller stream of results that are the most novel, important, and correct. Without explicit review, your options would be to either waste your time wading through the mess or use reviews from sources whose advice you consider on an ad hoc basis. I'm sure there are better systems that take advantage of the Internet, the wisdom of crowds, data aggregation and mining, etc. The current system of peer review does adequately perform a major service, though.
"my brain hasn't collapsed from the inrush of poorly filtered information" Of course not. However, you are either wasting a ton of time or you are using some other indirect method to decide what papers are useful to read. I'd love to hear if you think your personal approach to filtering non-reviewed content is superior to relying on peer review.
You've assumed that the non-reviewed content I'm reading is in the form of papers. It isn't; it's blog posts and code.
I can count on one hand the number of papers I've read in the past month or two describing an implementation of a technique where the implementation described was available publicly.
so... less than 32 papers have the implementation available publicly?Joking aside, how many papers have you read in the past month or two? I assume the number is at least in the double digits.
I also don't believe that the current conference / journal system for publication of results works especially well. There's a well-documented bias against the publication of negative results. It encourages authors to "save up" interesting work for publication when pieces of it could have been independently examined much sooner. In combination with "publish or perish" metrics, it discourages valuable or interesting work that might not ever rise to the level of a full journal publication. The format is incredibly stifling; important information is left out to satisfy page limits, code mangled to fit within columns, and interesting data visualizations not even considered because of the constraints of the media.
I think it's inevitable that the current common publication system will be recognized as a relic of the paper era, but I fear that any fix will only make superficial changes to the system. I think it'd be much more useful to have a publication system that allows work to be published with full data in a format that allows evolution in response to open reviews. This obviously wouldn't work well in areas where protection of patient privacy requires data obfuscation; I don't have a good answer for that. In all other areas, especially where the research is funded by public money, full data and code transparency ought to be key, especially where raw data must be adjusted or normalized in order to be useful and in situations where bugs in modeling programs could subtly affect results.
Edit: see also this excellent article from last month's Atlantic: http://www.theatlantic.com/magazine/print/2010/11/lies-damne...