News for hackers. Stuff that matters.
Re:Show HN: Spider Pro – easy and cheap way to scrape the in... by impatientduck on Thursday October 10, 2019 @01:03PM (#21216838)
Nonsense - beautiful soup is free.

Don't pay for something you can program yourself.

Re:Show HN: Spider Pro – easy and cheap way to scrape the in... by penagwin on Thursday October 10, 2019 @02:15PM (#21217558)
Disclaimer: I have nothing to do with this project, I'm also a programmer isn't part of it's target demographic. But:

Are you suggesting that if somebody wanted to setup a basic scrapper, they should learn how to install python, use packages, figure out how the webpage is structured, and dig through the page source to figure out how to extract their data?

This project has a decent visual data picker, interface, etc. You could literally hand it to a non-programmer, and have them scraping comments off hacker news without any programming knowledge at all.

I get paid over 25$/hr professionally as a programmer. This project costs 40$ one time. I'm not saying it's worth the money, but all it has to do is save somebody 2 hours of time.

If you are a programmer then this project likely isn't for you.

Re:Show HN: Spider Pro – easy and cheap way to scrape the in... by Cyberdog on Thursday October 10, 2019 @01:07PM (#21216885)
Sure, everyone's time is free too, after all.

(Alternate smart-ass reply: Surely you coded the browser, operating system, and network stack involved in posting this response by yourself, right?)

Re:Show HN: Spider Pro – easy and cheap way to scrape the in... by nirvdrum on Thursday October 10, 2019 @01:20PM (#21217026)
I'm not the OP, but it looks like it was in reply to the "cheapest way" part of the title. Incidentally, that line doesn't appear on the web site.
Re:Show HN: Spider Pro – easy and cheap way to scrape the in... by JustFinishedBSG on Thursday October 10, 2019 @01:44PM (#21217284)
> Surely you coded the browser, operating system, and network stack involved in posting this response by yourself, right?)

No because they were all free.

And I suck too.

But it was free.

Re:Show HN: Spider Pro – easy and cheap way to scrape the in... by sillydinosaur on Thursday October 10, 2019 @01:18PM (#21216997)
https://www.crummy.com/software/BeautifulSoup/bs4/doc/

I mean...

soup.find(id="link3") # <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

Is this hard?

Re:Show HN: Spider Pro – easy and cheap way to scrape the in... by Cyberdog on Thursday October 10, 2019 @01:22PM (#21217045)
For HN's audience, no, that's not hard. But this product is clearly not only for us, and you've also cherry-picked a very simple example.

There's also the aspect of downloading the page in the first place and dealing with things like authentication and bot detection which a product like this helps solve.

I personally don't have a use for this product right now, but I won't be so bold as to say I'll never find a case where using it wouldn't be easier or more cost-effective than hacking up my own solution.

Re:Show HN: Spider Pro – easy and cheap way to scrape the in... by slig on Thursday October 10, 2019 @03:46PM (#21218469)
Good, now do that on a page that uses JS and authentication. Trivial on this extension, not so much using BS.
Re:Show HN: Spider Pro – easy and cheap way to scrape the in... by penagwin on Thursday October 10, 2019 @02:18PM (#21217582)
If you show that to non-programmers they'll take one look at that and understand that about as well as egyptian hieroglyphics.

"Soup? Python? What's an 'id'? class? href? I just want to select the title!"