I don't think they can go back to scraping either, but I'm not certain of that aspect of it.
This may be the real reason behind the change. Perhaps CL just doesn't need it any more.
Also, CL has not removed itself from search engines. They're using NOARCHIVE.
User-agent: *
Disallow: /cgi-bin
Disallow: /cgi-secure
Disallow: /forums
Disallow: /search
Disallow: /res/
Disallow: /post
Disallow: /email.friend
Disallow: /eaf
Disallow: /reply
Disallow: /?flagCode
Disallow: /ccc
Disallow: /hhh
Disallow: /sss
Disallow: /bbb
Disallow: /ggg
Disallow: /jjj
Disallow: /*rss$
http://www.craigslist.org/robots.txt'jjj' is jobs, 'ggg' is gigs, 'bbb' is services, 'sss' is for sale, 'ccc' is community, 'res' is resumes.
http://newyork.craigslist.org/aap/ http://newyork.craigslist.org/roo/ http://newyork.craigslist.org/sub/
Looks like the same principle applies to a number of other subcategories. So while overview indices aren't spiderable, subindices are.
This is akin to freelance contract work: if you're a news photographer, there's usually a clause saying that while you can own your photos, you may not sell them to your client's competitors (at least right away).
So CL is saying to its users: Go ahead and copy and paste your stuff to other sites. So, not a huge difference by any means in what most users assumed.