News for hackers. Stuff that matters.
Re:Google Now Fills Out Forms & Crawls Results by ambition on Friday April 11, 2008 @10:51PM (#161591)
I noticed this happening in my blog's logs months ago, for my blog's search form. I thought it was fascinating. The keywords it chose to search were generally meaningful words that occurred frequently on the site. I'm not sure if they were correlated with the Google search terms that hit my blog.

The corresponding SEO tip is to make sure that your site's search feature works, delivers good results for the keywords and delivers the results to which you want Google to pay attention.

Re:Google Now Fills Out Forms & Crawls Results by rms on Friday April 11, 2008 @05:55PM (#161404)
Wow, this is kind of a big deal. Crawling the deep web seemed like the one real feature Google's vaporware competitors could tout.
Re:Google Now Fills Out Forms & Crawls Results by slapshot on Friday April 11, 2008 @08:41PM (#161535)
The deep web isn't limited to GET requests and forms that don't require passwords.
Re:Google Now Fills Out Forms & Crawls Results by rms on Friday April 11, 2008 @09:16PM (#161553)
Yes, but this is a start... give it a couple years.

And this is still more than the vaporware competitors that are still vaporware.

Re:Google Now Fills Out Forms & Crawls Results by phaedrus on Saturday April 12, 2008 @02:50PM (#161972)
I propose a new sport - Google Crawler hunting. Kind of like Snipe hunting, only you lay traps. Object is to see how long you can keep the Google bot there by generating new pages programmatically, and how many unique things you can get the bot to do.
Re:Google Now Fills Out Forms & Crawls Results by raghus on Saturday April 12, 2008 @12:56AM (#161660)
Here's Matt Cutts explaining it a little more: http://www.mattcutts.com/blog/solved-another-common-site-rev...
Re:Google Now Fills Out Forms & Crawls Results by michaelneale on Friday April 11, 2008 @06:41PM (#161444)
So its ok for them to do this? but if you do the same against google (scraping results) - presumably Bad Things happen?
Re:Google Now Fills Out Forms & Crawls Results by cstejerean on Friday April 11, 2008 @07:19PM (#161476)
Is their search url listed in their robots.txt?
Re:Google Now Fills Out Forms & Crawls Results by thorax on Friday April 11, 2008 @07:41PM (#161491)
Seems strange that you wouldn't have just looked when asking that question (less characters to type at least). Lots of things disallowed.

http://google.com/robots.txt

Re:Google Now Fills Out Forms & Crawls Results by Hexstream on Friday April 11, 2008 @11:07PM (#161597)
I think it was a rhetorical question.
Re:Google Now Fills Out Forms & Crawls Results by michaelneale on Sunday April 13, 2008 @12:45AM (#162220)
Yes it was. But I should have been clearer for the type of literal argument starters that would tend to hang around here ;)

I wonder if google always honour robots.txt

Re:Google Now Fills Out Forms & Crawls Results by Hexstream on Sunday April 13, 2008 @09:46AM (#162396)
I hope so because otherwise they could get stuck in one of those:

http://en.wikipedia.org/wiki/Spider_trap

Re:Google Now Fills Out Forms & Crawls Results by flyhighplato on Friday April 11, 2008 @08:02PM (#161509)
What an interesting problem... so, if you were to build a searchbot, what would it type into those form fields it found?

It has to be very difficult, especially since you can't do too many submits. Could you somehow infer what to put in there based on searches in your own database already?

Re:Google Now Fills Out Forms & Crawls Results by dzorz on Saturday April 12, 2008 @08:35AM (#161809)
What happens when google starts editing wiki pages and filing new bug reports/tickets?
Re:Google Now Fills Out Forms & Crawls Results by tocomment on Saturday April 12, 2008 @11:51AM (#161876)
It only submits GET forms, you shouldn't use those for those things.
Re:Google Now Fills Out Forms & Crawls Results by davidw on Saturday April 12, 2008 @12:15PM (#161888)
I was sort of hoping they had a valid credit card number:-)
Re:Google Now Fills Out Forms & Crawls Results by Youhavetobelogg on Saturday April 12, 2008 @03:33AM (#161744)
text Google Now Fills Out 1 minute ago link
Re:Google Now Fills Out Forms & Crawls Results by danteembermage on Saturday April 12, 2008 @09:53AM (#161830)
scaling Ron Paul Graham