Re:Good News: Craigslist drops exclusive license to your posts
by jobu on Thursday August 09, 2012 @08:36PM
(#4364179)
That sucks, searching is how I double-checked dubious craigslist posts. Select a couple phrases and search google. If identical posts pop up under various other cities on craigslist, then I knew for sure it was spam/fraud.
Re:Good News: Craigslist drops exclusive license to your posts
by IanDrake on Friday August 10, 2012 @12:40PM
(#4366944)
Just one of the many ways third parties fix broken things on Craigslist. Things Craigslist refuses to recognize.
Re:Good News: Craigslist drops exclusive license to your posts
by danso on Thursday August 09, 2012 @06:23PM
(#4363619)
I think this is correct, though the EFF post is too vague, and doesn't quote the passage in its current state.
Also, CL has not removed itself from search engines. They're using NOARCHIVE.
Re:Good News: Craigslist drops exclusive license to your posts
by dredmorbius on Thursday August 09, 2012 @06:31PM
(#4363667)
Last I checked, robots.txt excludes all crawlers from at least several major classifications, including housing listings (/hhh):
User-agent: *
Disallow: /cgi-bin
Disallow: /cgi-secure
Disallow: /forums
Disallow: /search
Disallow: /res/
Disallow: /post
Disallow: /email.friend
Disallow: /eaf
Disallow: /reply
Disallow: /?flagCode
Disallow: /ccc
Disallow: /hhh
Disallow: /sss
Disallow: /bbb
Disallow: /ggg
Disallow: /jjj
Disallow: /*rss$
http://www.craigslist.org/robots.txt'jjj' is jobs, 'ggg' is gigs, 'bbb' is services, 'sss' is for sale, 'ccc' is community, 'res' is resumes.
Re:Good News: Craigslist drops exclusive license to your posts
by danso on Thursday August 09, 2012 @07:02PM
(#4363813)
Right...but not apartment listings (aap)
http://newyork.craigslist.org/aap/ http://newyork.craigslist.org/roo/ http://newyork.craigslist.org/sub/
Re:Good News: Craigslist drops exclusive license to your posts
by dredmorbius on Thursday August 09, 2012 @07:20PM
(#4363882)
Point. I mistakenly assumed /aaa covered all of housing.
Looks like the same principle applies to a number of other subcategories. So while overview indices aren't spiderable, subindices are.
Re:Good News: Craigslist drops exclusive license to your posts
by defen on Thursday August 09, 2012 @08:15PM
(#4364114)
Google still provides a screenshot of CL pages - I doubt it would take much work for an enterprising hacker or 3taps to hook an OCR library up to that.
Re:Good News: Craigslist drops exclusive license to your posts
by SilasX on Thursday August 09, 2012 @08:23PM
(#4364153)
But wouldn't that be insane on bandwidth?
Re:Good News: Craigslist drops exclusive license to your posts
by rabidsnail on Thursday August 09, 2012 @09:02PM
(#4364262)
Inbound bandwidth is free.
I don't think they can go back to scraping either, but I'm not certain of that aspect of it.
This may be the real reason behind the change. Perhaps CL just doesn't need it any more.