News for hackers. Stuff that matters.

A comment on Brandchirps – Simple Brand Monitoring.

Re:Brandchirps – Simple Brand Monitoring by fenwick67 on Sunday January 19, 2020 @12:53PM (#22092744)
One thing that might hook me is if you said which sites are scraped.
[ Parent | on HN ]
Re:Brandchirps – Simple Brand Monitoring by JamesGreene on Sunday January 19, 2020 @07:32PM (#22095226)
I know a lot of people are asking for this, but it would literally fill up this whole page listing all the URLs.

Best way I can describe it is "publicly allowable URLs on the web" which would include blogs, forums, social networks, websites, and more. If we can pick up the HTML/RSS/ATOM/Json/text.. then we try to get it.

We don't scrape any sites that disallow it in their robots txt and we don't scrape material only available behind logins and paywalls.

[ Parent | on HN ]