News for hackers. Stuff that matters.
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by noelchurchill on Wednesday April 16, 2008 @01:37PM (#165355)
why not block all search engines?
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by comatose_kid on Wednesday April 16, 2008 @01:50PM (#165378)
Because News.YC doesn't have it's own search engine. So I use google to search the site.
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by diego on Wednesday April 16, 2008 @03:00PM (#165486)
Why not solve the real problem and set up a search engine for News.YC? This has other advantages such as controlling what shows up in the results, different ways to sort them, etc.

It's easy to do with open-source software such as Solr (lucene.apache.org/solr) or Hounder (hounder.org, shameless plug because it is our product), which powers the wordpress.com site.

Edit: it looks like someone already did this: searchyc.com.

Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by rglullis on Wednesday April 16, 2008 @03:43PM (#165543)
You must be new here.

The day PG & Co adopt some piece of software that is made in Java... well, I can't even finish the sentence.

Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by brlewis on Wednesday April 16, 2008 @04:01PM (#165561)
Part of the & Co is Paul Buchheit, who wrote some of the news.yc javascript. Paul Buchheit's company friendfeed is using lucene for search.
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by rglullis on Wednesday April 16, 2008 @04:14PM (#165576)
i didn't know that.

Either way, I think that PG is the king of guy who would rather work on his very special design (staying in Arc-land as long as possible) than adopting systems where you no longer have control.

Letting Paul work on the JavaScript makes sense, in this case. PG has no control over client-side code, so he prefers to not bother with it.

Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by mynameishere on Wednesday April 16, 2008 @06:18PM (#165719)
Google uses Java, and since that is evidentally the only acceptable search engine, I'm not sure pg's posited elitism really goes that far.
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by rglullis on Wednesday April 16, 2008 @06:28PM (#165727)
I didn't say that PG would refuse to use any system that was developed in Java. What I meant that he would never would put himself in a situation where the application would depend on Java for development, or to add more features.

To integrate with Google, you talk HTTP, not Java. To adopt Lucene, you'll have to talk Java. There is quite some difference, isn't it?

Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by gojomo on Wednesday April 16, 2008 @07:46PM (#165788)
You can push everything you want indexed into SOLR by HTTP -- so even though it's Java/Lucene you don't have to get your hands dirty.
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by rms on Wednesday April 16, 2008 @09:08PM (#165843)
They use some Java, but a whole lot more Python and C++...
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by gus_massa on Wednesday April 16, 2008 @09:41PM (#165868)
Usualy Google gets better results that a custom search engine for your site, because Google also knows the links form other web pages to your site.
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by nreece on Wednesday April 16, 2008 @07:34PM (#165776)
Did you not find searchyc.com useful?
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by kirubakaran on Wednesday April 16, 2008 @01:51PM (#165380)
Googling for my name lists my Hacker News profile as one of the top results. I think that is really cool.
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by bgutierrez on Wednesday April 16, 2008 @03:18PM (#165510)
That is cool. My username returns my profile here as the number one hit. I had no idea it would score so highly.
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by dangoldin on Wednesday April 16, 2008 @10:30PM (#165917)
Not as cool as mine. My username returns a wikipedia page about some asteroid. Cool stuff.
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by jbrun on Wednesday April 16, 2008 @10:32PM (#165921)
googling my username brings up my hacker profile too
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by msg on Wednesday April 16, 2008 @01:50PM (#165379)
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by kirubakaran on Wednesday April 16, 2008 @01:51PM (#165383)
> There's no other search.

http://www.searchyc.com

(edit: I'm not the one who voted you down)

Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by jsrn on Wednesday April 16, 2008 @01:54PM (#165390)
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by aston on Wednesday April 16, 2008 @04:23PM (#165592)
They stopped indexing looong ago.
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by jsrn on Thursday April 17, 2008 @05:40AM (#166204)
thanks, I didn't know that! Perhaps they should say so on the page...
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by jasonyan on Thursday April 17, 2008 @07:03AM (#166224)
Done :)
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by kirubakaran on Thursday April 17, 2008 @11:21AM (#166389)
Hey can you publish the source code of the search tool?
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by jsrn on Thursday April 17, 2008 @12:21PM (#166459)
wow, that was quick:-)
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by alex_c on Wednesday April 16, 2008 @02:41PM (#165464)
Does that actually work for people? I've tried using Google to search News.YC for an older topic on a couple of occasions, and the results have been very hit-or-miss. Is it just me?
Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by kirubakaran on Wednesday April 16, 2008 @02:55PM (#165481)
(plug)

If you remember some key words from the title, you can do a quick find in http://www.kirubakaran.com/phr0zen (snapshots of front page). It is not really search search but I find it useful all the time.

Re:Poll: Should this site be visible to Yahoo, MSN crawlers? by rokhayakebe on Wednesday April 16, 2008 @03:04PM (#165495)
right on. then implement a search feature