You are not entitled to scrape content from other sites, even if they contain publicly submitted data. I do support this sort of thing in the hacker spirit (as long as you aren't being too agressive) but selling an application using these techniques is downright unethical.

You have zero control over what you are selling to people. They don't know that you aren't using an offical means to get the data and the application could stop working at any time. And if you were scraping via the client (as opposed to collating data on a server of your own) you also put your customers in a position where they might be violating the site's TOS and risk reprecussions (ip ban, etc).

> You are not entitled to scrape content from other sites, even if they contain publicly submitted data.

Isn't this Google's business model? They take snapshots of the public web and show organized and relevant snippets. Last time I checked, they never asked me for permission.

They don't ask your permission, but they do respect robots.txt files as well as noindex, nofollow.

I am sure the orignal author does as well. This is more a question of "fair use", not robots.txt

Indeed - and i believe scraping data off a publically accessible webpage (e.g., one where you do not require registration and login/password) falls under fair use, provided you do not take up more bandwidth/resources than the average user of that site.

I recall there being some precedent for this sort of fair use - something like a phone directory - the information is not copyrighted, but the arrangement and layout is. So hence, you can't just iframe a site and present it, but obtaining the data, and deriving a new work from it should fall under fair use.

That court case was Feist v. Rural. The court ruled that copying the listings in a phone book was permitted because facts cannot be copyrighted.

And they can be asked to remove listings from their search. Ask rupert Murdoch

> a) People go to craigslist and create content about some aspect of their lives. They, as the original authors of the content, post it to craiglist with the intention that the general public can see it.

Craigslist users only agreed to have their content displayed on the Craigslist site, not on any other random site on the Internet who wants to relist their posting without the users explicit permission.

As a regular Craigslist user (buyer, seller and message board poster), my personal preference is to have material I post only accessible directly from CL. In other words, I do not want my classified ads, contact information contained therein, pictures, social banter, business ads and other information available to any external information service or website.

This is my expectation not only of CL, but of many sites. I understand that others may disagree. But, I suspect that my expectation of non-propagation of material is the default expectation for many or most sites that accept user submitted material.

If CL is somehow forced to let others scrape their content, I would welcome an option to turn on a switch that will block/hide/protect the material I contribute from external entities. On second thought, I would rather have the do not share option be the default and let me opt in to sharing everything always, share by material classification or one post at a time.

I would expect that many users would feel this way.

I can guess why. But could you tell us specifically?

Assume for the sake of the question that whereever the data may appear on the web, it would always have a "Source: " line to indicate its original source, i.e. the site to which you submitted it. Assume that the integrity of the data could also be verified, e.g., through a digital signature.

A few reasons came to mind when I originally wrote it. I know that there are many tens of other reasons why it would make good sense to keep CL content CL-exclusive.

Of course, I realize that this must be tempered with the realization that there are situations where scraping CL and re-posting the information on another site may make sense for the original poster (e.g., a hotel ad).

Here are a few of the negative items that come to mind for just classified ads:

- When I complete a transaction and delete a CL ad, I do not want other interested parties calling, texting and e-mailing me hours, months or years afterwards

- I already dislike the 10-to-1-ish ratio of SPAM/scammer/nutjob banter to communications from legitimately interested parties. I suspect any proliferation of my postings beyond CL will only tend to increase this ratio.

- The text and images within my classified ads may be more easily accessible to scammers who just need some content to farm and post elsewhere (e.g., other classified forums, eBay, AirBnB, etc.)

- Sites which do the automated scraping of CL can directly be or can become a target and/or searchable repository for intelligence gathering and source for creating new or expanding existing personal identification databases

There are countless other scenarios ranging from good to bad regarding this subject. For now, I'm convinced that the potential for the bad, the incompetent, the annoying, the negligent, the criminal and other negative scenarios outweigh the good. This is really meant to apply to how I use CL. As I mentioned before, there are undoubtedly specific scenarios where many of the potentially negative items would be moot for specific CL users.

Obviously, there are other factors here too such as the commercial viability/legality/precedent of allowing non-CL entities to use CL resources with or without restriction to directly or indirectly profit. From my perspective as a user, it's not really that important other than an arguable matter of principle.

I think most Craigslist sellers want as many people as possible to see their listings. Your criteria exclude Google, which indexes every publicly available user-generated content site and thereby benefits Craigslist sellers.

That was CL's big mistake. These sort of unreasonable licensing terms that users never read are exactly the kind courts refuse to enforce.

Maybe you just want CL to win? That's different from whether CL has a real chance of winning. This case is unlikely to go to trial. That's because CL has a lot to lose and 3Taps has very little to lose. Some believe the reason there are few lawsuits on these issues that are not settled out of court is because websites like CL are afraid of having a court decide on them. Why would they want to avoid having a court decide?

I think you're talking in two directions here.

'any other random site on the Internet' is hardly the same as a single-purpose (native) app for making CraigsList listings easier to parse. I'm sure that most of the random Internet sites might not even be attributing the data back to CraigsList.

When it comes to apps like this, all Craigslist has to do to compete is make their UI for browsing/tracking listing better. If their interface was better, then people wouldn't feel the need to pay ~$3 for a separate app to browser their listings.

I see what you're saying and that may technically be true.

I'm not really arguing the legality of the issue though. I'm just talking about the spirit of what the content authors were trying to accomplish which I believe is: have some random person on the internet see their post and, subsequently, arrive at some sort of mutually beneficial arrangement with that person.

There's any number of places on the net where someone could post content and "have some random person on the internet see their post" ... but they continue to choose Craigslist because it's got the audience. If they want more-broad usage rights for what they want to post, they're free to chose an alternative platform on which to post.

This argument is what the monopolist uses when they are faced with regulation. It isn't correct, because the user _isn't_ really going to be able to post on another forum.

This is just like if telcos say 'go to some other provider if you don't like my terms and conditions'. except, they have a monopoly.

Just because CL has a large user base doesn't make it a monopoly. CL can go the way of MySpace if a better competitor comes along. There isn't any barrier to entry to create your own CL + Padmapper type listing site that requires some sort of monopoly break up.

The hard part for you is creating a compelling enough competitor 15 years too late and also to make it worth people's while to start using your site vs. CL.

> The hard part for you is creating a compelling enough competitor 15 years too late and also to make it worth people's while to start using your site vs. CL.

which is essentially why CL has a defacto "monopoly" - there is no way someone can come and make a different CL.

Yes, but it's definitely true that they understand that to mean public, and it's almost certainly true that they would rather have more eyes than fewer. It sucks that that isn't a legally compelling argument and this guy will wind up being shut down though.

I fundamentally disagree.

The users know CL is sharing it with the world and that's what the users expect. As the article says, CL is happy for search engines to access these. Just obviously not search engines that they don't like.

You don't get to pick and choose who accesses your website unless you put a password up. And if the content's not copyrightable...

If I place an ad on craigslist, I would like to know if my ad will show up on...SanFranGayApts.net or something like that too. Where did I agree to have MY ad syndicated? Nowhere, that's where.

And why would ad copy not by copyrightable? I don't see a creative commons license at the bottom of craigslist ads.

I'm not sure where all this entitlement comes from that you think you have a right to another company's database simply because it's index is public.

If you wanted your personally identifiable data protected from public access (being cached by search engines), then presumably you would not submit it to a web site that is open to the public (not password protected).

If Craigslist changes its name or sells its SF apartment listings to some site with a name you do not like, what can you do? Answer: Nothing, because you gave them an exclusive nonrevocable license to your data. If you really wanted control over your personally identifiable data then it would probably be wise not to submit it to a site that gets scraped and cached by Google (from which anyone can make fair use) and certainly not under a license that gave you no control over the data after it's submitted. You would want a site that has some sort of protections like passwords, that allows some degree of pseudo anonymity so your name is not embedded in an evergreen search engine cache; and you would want the right to tell the site to remove/delete the data at your direction, if that became necessary (e.g. in your example scenario).

In short, Craigslist really isn't in the business of protecting your data for you. Its terms are what they are because it's seeking to protect its own business, which relies on your data.

Craigslist's copyright claim is likely indefensible. Read the article for the reason why.

> it's called Craigslist Commando

Issues over content ownership aside, calling it that is a pretty clear trademark violation.

I'm as much about free access to data as anyone, but holy shit do you have a warped view of the world.

Craigslist doesn't have to pay money to run their company and users don't have to use Craigslist.

But because Craigslist does and the users do, doesn't give anyone the right to scrape this data. The individual user owns their content, but Craigslist owns and runs the servers and access to this data for third parties is not a right -- it's a biz dev deal or partnership.

If the world worked how you want it to, no one would run any services because the load generated from scrapers would outweigh real user usage by a factor of 100x. And your competitors and parasitic freeloaders would just DoS you into monetary ruin.

You're not entitled to anything, and Craigslist doesn't owe you a business model.

TL:DR: This lawsuit is going straight into the trash.

People should post their cease and desist letters from CL so others can see. I doubt you were the only person to receive one. Sometime when something doesn't seem right, it's because it isn't.

I also received a C&D two weeks ago via email and FedEx for my craigslist interfacing iOS app. I also got an offer for a license but haven't heard anything since Thursday.

Hey, could you email me? [email protected]

Us little guys need to stick together ;).