News for hackers. Stuff that matters.
Re:Mapping CraigsList Housing Ads by coglethorpe on Friday February 27, 2009 @05:20PM (#497737)
I hate to be a wet blanket, but didn't http://www.housingmaps.com/ do that like years ago?
Re:Mapping CraigsList Housing Ads by andrewljohnson on Friday February 27, 2009 @05:21PM (#497739)
Read my blog post above. I acknowledge them and talk about the improvements we made.

For example, HousingMaps.com maps a couple dozen cities, and we map all 326 US cities.

Re:Mapping CraigsList Housing Ads by oldgregg on Friday February 27, 2009 @07:26PM (#497827)
I thought I read somewhere that housingmaps had a special agreement with craigslist... But I hope you stick around because the lack of cities on hm is really annoying.

Craigslist really needs an API... and global search. PLEASE.

Re:Mapping CraigsList Housing Ads by jonmc12 on Friday February 27, 2009 @05:42PM (#497755)
First, cool. Next, can you share your crawler code?

Lastly, this violates craiglist's usage policy right? Did you research the implications and what did you find?

Re:Mapping CraigsList Housing Ads by lacker on Friday February 27, 2009 @06:04PM (#497782)
This does violate craigslist's usage policy.

craigslist permits you to display on your website, or create a hyperlink on your website to, individual postings on the Service so long as such use is for noncommercial and/or news reporting purposes only (e.g., for use in personal web blogs or personal online media). If the total number of such postings displayed or linked to on your website exceeds one hundred (100) postings, your use will be presumed to be in violation of the TOU, absent express permission granted by craigslist to do so.

Re:Mapping CraigsList Housing Ads by andrewljohnson on Friday February 27, 2009 @05:55PM (#497772)
First, I don't make a habit of reading ToS, so I don't know if I'm in violation or not :)

Second, I posted the code here. It's kind of a mess... http://www.trailbehind.com/trips/view_report/116

Re:Mapping CraigsList Housing Ads by andrewljohnson on Friday February 27, 2009 @05:20PM (#497738)
For those interested in the project, I also wrote up a short blog post about it at: http://www.trailbehind.com/trips/view_report/115

Any feedback or bug reports you have are much appreciated.

Re:Mapping CraigsList Housing Ads by Cooldude on Friday February 27, 2009 @06:57PM (#497816)
Good job, I'll be looking for a place soon! I have a few question/comments:

- Open links in a new window. - Have a date picker or a dropdown with 1 day ago, 2 days ago, etc. - Where are the nearby photos near? - Show photos button, why is it different than the checkbox for dogs/cats? Its kind of confusing. - Change it so when you click from list of adds, the map moves to the ad's location on the map. - When you click on an ad on the map you place a pin on the map. What can you do with this? I'd rather see pics/summary information and then I would decide whether I want to save this ad.

Re:Mapping CraigsList Housing Ads by andrewljohnson on Friday February 27, 2009 @07:19PM (#497824)
All good suggestions

1. I implemented open in new window 5 minutes ago

2. I'll add my calendar widget to replace the data selector

3. I'll overlay the polygon of the photo search area.

4. Good idea for images for cats/dogs.

5. I had it set up to pan to the point originally, but now I just open a new window, so I'm not sure that's useful anymore.

Re:Mapping CraigsList Housing Ads by rwrice on Friday February 27, 2009 @06:27PM (#497793)
This is good stuff.I like how the list of ads changes as I pan around the map. What else does Trailbehind do?
Re:Mapping CraigsList Housing Ads by mikeyur on Friday February 27, 2009 @06:34PM (#497798)
Suggestion: Open ads up in a new window, when I click back after viewing an ad I have to re-select city, etc.
Re:Mapping CraigsList Housing Ads by andrewljohnson on Friday February 27, 2009 @06:55PM (#497815)
Thanks for the suggestion! I implemented it :)
Re:Mapping CraigsList Housing Ads by davidaue on Friday February 27, 2009 @07:50PM (#497840)
It seems like your site is not showing the majority of the adds. If I look at my neighborhood (El Cerrito, CA) there are hundreds of listings in the last week on craigslist but setting the cutoff date to 2/20 on your site only displays 3 or 4 results in the same area.
Re:Mapping CraigsList Housing Ads by andrewljohnson on Friday February 27, 2009 @08:17PM (#497863)
Sorry, in the future we'll have ads going back further, but we didn't start crawling CraigsList until the 20th.

Each day the bot runs is another day we have stored.

Re:Mapping CraigsList Housing Ads by davidaue on Friday February 27, 2009 @10:33PM (#497950)
I think you're missing what I'm saying. If you look at San Francisco/East Bay/Albany:El Cerrito on craigslist there are 30 listings for 2/27. If I look in that area on your map I see only 2 listings.
Re:Mapping CraigsList Housing Ads by sam_in_nyc on Saturday February 28, 2009 @02:42AM (#498084)
Same here for my neighborhood. For me, this warrants not using the product at all... why use something that has an incomplete dataset? I could be missing out on that perfect listing!

Another thing is that some postings don't contain sufficient information for a machine to know its location... such as "near the Xyz subway station" or other atypical types of addresses that the project's parser might miss.

It's a cool mash-up, but I'll pass.

Re:Mapping CraigsList Housing Ads by andrewljohnson on Saturday February 28, 2009 @04:35PM (#498503)
Ok, I see your point.

For some ads, there is no geo-data. To fix that for the next release, we'll list ALL ads, not just the mappable ones, in the Ad Widget. And we'll indicate in the table if the ad is on the map.

One good side effect though is the ads on the map tend to be less spammy than the non-mapped ads. I assume this is because the spam ads often embed their spam in HTML and don't use CraigsList's structure well. A good landlord is going to make sure their ad links to a Google Map on CraigsList, which also means we will index them.

Re:Mapping CraigsList Housing Ads by sam_in_nyc on Saturday February 28, 2009 @02:46AM (#498086)
Not allowing me to enter my location, and instead having me zoom/wait-to-load several times makes me wish that closing the tab somehow inflicted pain on the software.
Re:Mapping CraigsList Housing Ads by vaksel on Friday February 27, 2009 @06:00PM (#497779)
how come half the kansas and half of nevada is missing?
Re:Mapping CraigsList Housing Ads by psadauskas on Friday February 27, 2009 @06:07PM (#497784)
Nobody lives there? There probably isnt a craigslist to cover those areas.
Re:Mapping CraigsList Housing Ads by andrewljohnson on Friday February 27, 2009 @06:08PM (#497786)
Are you sure about that? I think we have all the cities that CraigsList maps in the US.

Does Nevada have more than Reno and Las Vegas? We have both of those, in opposite corners of the state.