[1] http://www.searchlores.org/yoyo1.htm (try to read past the preachy anti-commercialism, he could get kind of hot about that--whether you agree or not, there's troves of knowledge to be gained from that site)
http://channel101.com/ offers a nice selection of "unpopular" shows. You should check Danny Jelinek's "Everything" (https://vimeo.com/channels/231109) if you're into video-art.
Anything you can identify as pure SEO spam, exclude of course. But if it's some original content that just isn't well connected or whatever, include it.
Then just observe user behavior for a while until you can discern patterns in how people who play around with that use it to discover new and interesting stuff. Maybe a new algorithm might come of it.
You need to either remove the top 100 (or 1000 etc) and look at what remains, or reverse only the top 10000 results (or 1000 etc).
This website is an interesting experiment however if you're after discovering random semi related pages, why not just use google or stumble upon?
Is there a way to include domains automatically excluded? For example WordPress.com is excluded.
Disclaimer- former WREK DJ. :-)
"Artists who meet any of the following criteria are automatically disqualified from airplay on WUOG 90.5FM:
The artist is or has been in regular rotation on major commercial radio stations. This does not apply to material used in specialty shows on major commercial stations.
The artist has a music video that has aired on major music video stations. This does not apply to material used in specialty shows on major commercial networks.
Another criterion that DJs must use when selecting music for airplay is its position on the Billboard Album Charts. If one of the artist’s albums has entered the top 50, then a DJ cannot play that artist."
http://lesswrong.com/lw/2pv/intellectual_hipsters_and_metaco...
Of course, there are great websites out there that are very popular (Wikipedia, NYTimes/WSJ, StackOverflow). I'd love to see a search engine with a better signal for quality than non-popularity (this search engine), or SEO (Google), but it's a fun start. :)