News for hackers. Stuff that matters.
Re:Show HN: I built a service to discover and monitor rapidl... by saturday14 on Saturday April 04, 2020 @09:38AM (#22778141)
Is this based on Google Trends? Where is the data coming from?

Interesting project, btw

Re:Show HN: I built a service to discover and monitor rapidl... by mikerubini on Saturday April 04, 2020 @09:39AM (#22778144)
Our data relies on Google and it is based on what consumers are searching for.

Specifically, it relies on Google Trends, Google News, Google searches, Amazon searches, Twitter searches and YouTube searches (for now!)

We also have a partner for e-commerce data (usecart.com)

Re:Show HN: I built a service to discover and monitor rapidl... by saturday14 on Saturday April 04, 2020 @09:43AM (#22778167)
Very cool. I guess as you grow, you could get more partners and the data will keep getting more and more accurate. I wish big guys like Amazon released aggregated sales data.

How do you get access to Amazon searches, google searches etc? do you have to pay for those?

Re:Show HN: I built a service to discover and monitor rapidl... by mikerubini on Saturday April 04, 2020 @09:46AM (#22778189)
Absolutely. I don't pay to access data. All of my projects are around collecting data. I'm a master scraper ️

If you are curious, at https://rubini.solutions you can check out my other SaaS products =)

Re:Show HN: I built a service to discover and monitor rapidl... by saturday14 on Saturday April 04, 2020 @10:22AM (#22778362)
Just checked your list of products, very cool!

I've always wondered about building a career with scraping, but never got around to it. One reason is the legality of scraping - big sites like Craigslist have gone after (and shut down) smaller projects. I believe priceonomics (YC company) built a big business with scraping.

Re:Show HN: I built a service to discover and monitor rapidl... by riku_iki on Saturday April 04, 2020 @10:48AM (#22778515)
And how do you scrap Google data? They don't ban you or ask to enter "I am not robot" captcha?
Re:Show HN: I built a service to discover and monitor rapidl... by mikerubini on Saturday April 04, 2020 @12:35PM (#22779337)
No, as I mentioned, I'm a master scraper.
Re:Show HN: I built a service to discover and monitor rapidl... by saturday14 on Saturday April 04, 2020 @04:28PM (#22781243)
Are you not breaking their ToS though?
Re:Show HN: I built a service to discover and monitor rapidl... by mikerubini on Saturday April 04, 2020 @04:29PM (#22781252)
Nope, everything is ok