News for hackers. Stuff that matters.
Re:Ask HN: Side projects that are making money, but you'd no... by snird on Saturday June 06, 2020 @11:48AM (#23439963)
It's probably not what you meant, but I recently created a weekend project bot that scans facebook for apartments rental ads. Thanks to this bot, I landed an opportunity I wouldn't have catch otherwise, saving me 6,000$ a year. Assuming I will hold my new apartment for 3 years, this is a net savings of 18,000$. I haven't sold anything, but yet I created something that "generated money" for me.

My code is open source by the way, and I wrote about it here: https://snir.dev/blog/apartments-bot/

Re:Ask HN: Side projects that are making money, but you'd no... by HaloZero on Saturday June 06, 2020 @12:38PM (#23440382)
Interesting. I've never heard of apartment posts in facebook groups but I'm in the bay area.

Do other people here in the US have this experience as well? I was thinking it'd be Facebook marketplace, craigslist, and apartment sites (like apartments.com)

Re:Ask HN: Side projects that are making money, but you'd no... by snird on Saturday June 06, 2020 @05:33PM (#23442404)
I can say that people in Israel prefer facebook groups for many reasons:

1. It allows for "team search". My friends knew I'm looking, so they tagged me in relevant posts they saw. Can't do it anywhere else (including marketplace by facebook themselves)

2. It allows for group communication with the publisher of the post. One asks - everybody sees.

3. It gives a sense of power to those who seek to rent. Many posts with high prices have comments like "You are crazy", "You are a pig", "whoever pays this much is as sucker". It almost warms the heart, we fight the greedy landlords. It almost make it fun, like a witch hunt.

Re:Ask HN: Side projects that are making money, but you'd no... by bradlys on Saturday June 06, 2020 @09:43PM (#23444024)
As a landlord though, what’s the reason to use Facebook then? Is there no alternative like craigslist in Israel?

I’ve only seen posts of apartments for rent in Facebook groups that are aimed at college students. (In the US)

Otherwise, Craigslist is basically king.

Re:Ask HN: Side projects that are making money, but you'd no... by snird on Sunday June 07, 2020 @03:37AM (#23445517)
Young people use facebook to find apartments.

So if your apartments target young couples or room mates, facebook is your place. For families we have craiglist-like solutions.

Re:Ask HN: Side projects that are making money, but you'd no... by mbrameld on Saturday June 06, 2020 @01:28PM (#23440791)
A Facebook group is the primary source for advertising rental properties in Juneau, AK. Just a normal group, not the marketplace.
Re:Ask HN: Side projects that are making money, but you'd no... by stevekemp on Saturday June 06, 2020 @02:56PM (#23441467)
Ditto Helsinki, Finland:

https://www.facebook.com/groups/235368102105

I listed my rental flat a bunch of places before using the group, since I started using it I'd not look elsewhere.

Re:Ask HN: Side projects that are making money, but you'd no... by cedricium on Saturday June 06, 2020 @01:23PM (#23440755)
FB buying/selling groups (particularly for housing) are quite common I'd say. In my experience, you will generally find them categorized as part of a college or university (e.g. "San Jose State University (SJSU) Housing"), but a quick search for "Bay Area housing" also results in numerous such groups.
Re:Ask HN: Side projects that are making money, but you'd no... by asdff on Sunday June 07, 2020 @03:10AM (#23445406)
typical at college campuses in the us
Re:Ask HN: Side projects that are making money, but you'd no... by jack95 on Monday June 08, 2020 @04:51AM (#23454843)
Thanks for sharing this, I tried to write a bot to crawl a few music related Facebook groups a while ago and it was really difficult. I'm going to give it another shot following a similar method to this. Cheers!
Re:Ask HN: Side projects that are making money, but you'd no... by radihuq on Saturday June 06, 2020 @02:49PM (#23441426)
This is great, congrats on the savings. Do you have any opinions on the ethics of web scraping? I have some projects I want to pursue that involve web scraping but I'm worried about

1) any illegal technicalities 2) is it a dick move to the website owner

Re:Ask HN: Side projects that are making money, but you'd no... by snird on Saturday June 06, 2020 @05:24PM (#23442346)
I was a CTO of a company that did scraping to gather data and sell the analysis. We did this for sellers on Amazon.

In short about legality - it depends on the country. But most will define user content as owned by the user, not the platform, therefore legal to scrape and use, as the user intents for the data to be public.

But sites will fight you on this. Amazon specifically give up a really good fight.

Ofcourse, this talks about user content in an "open" platform. If the content is owned by a business that has no intentions for it to be open for everyone in every medium, its probably illegal.

In regard to being a dick move, it depends. If someone posted an ad in facebook, he intended for it to be seen. He doesn't care how. I just tunnel it to myself in easier way for me to consume.

Re:Ask HN: Side projects that are making money, but you'd no... by yowlingcat on Saturday June 06, 2020 @07:41PM (#23443345)
I think there was also case law about this recently[1] regarding LinkedIn, where if the content isn't behind a login wall, it can't legally be considered anything but public. The UGC aspect is an interesting angle I haven't thought of, but that's a fantastic point too. Would I be correct in saying that the same mechanism companies set up to evade responsibility for illegal UGC also sets it up to be legal to scrape? And if the user owns the material, what happens if you've already made money selling that data and they ask for their cut of the proceeds?
Re:Ask HN: Side projects that are making money, but you'd no... by mmcwilliams on Saturday June 06, 2020 @04:33PM (#23442015)
Not OP, but I find that the ethical questions tend to not be about web scraping itself and more what you are using it to accomplish. If you want to scrape and rehost some data then your ethical violation is in the copying and rehosting, not that you automated the downloading of data. It would be the same if you copied all the data by hand and then rehosted it.

Then, of course, the way you go about it is important, too. Sapping the resources and bandwidth of a small site by sending it thousands of requests a minute is very different from automatically checking a public index once an hour.

Re:Ask HN: Side projects that are making money, but you'd no... by tasuki on Saturday June 06, 2020 @05:45PM (#23442501)
1) No idea... probably location-specific too.

2) I ask the owner first (perhaps they even want to send me the data directly). If they don't respond, I go ahead and scrape, spreading the requests over a long time so as not hit their website too hard.

Re:Ask HN: Side projects that are making money, but you'd no... by ta17711771 on Saturday June 06, 2020 @03:30PM (#23441651)
> is it a dick move to the website owner

FB gets more user engagement out of this, which is their goal. No harm, no foul.