News for hackers. Stuff that matters.
Re:Show HN: CloudScrape – Cloud-based web scraping platform by zer0defex on Saturday October 17, 2015 @07:59AM (#10404144)
Nice timing, as just this week I had my first exposure to the Cloudscrape platform. Credit where credit is due, I found the platform refreshingly feature rich even at lower tier (i.e. free) account levels compared to other offerings from competitors.

Downsides were like so many other crawlers/spiders - too much in trying to meet the needs of all. Aiming for the 95% is easy, 95 to 99% though, not so much. Hence all the corpses littered along this path.

Recommendations:

Define solid, real-world examples of your platform in practice and really, no bullshit, what makes you different from every other platform that's 97% polished out there?

You didn't meet my needs because I needed much more in the way of parsing network requests from pages, mostly in handling if/then scenarios. A relatively simple scenario, to ME, and there-in lies the challenge of the crawler market, it's the last 5% that's custom to each scenario that so often delivers 98& of the value.

Suggestion: Identify and use the talent you obviously have to destroy some high value markets. Hint: Prioritize targets that still champion a desktop app at any point in the top 5 in terms of data collection.

Outside of that and far more niche (and subsequently far lower priority, allow for parsing / handling handling network requests and responses via custom app/API instead of just block all, some, or none)

Re:Show HN: CloudScrape – Cloud-based web scraping platform by henrik1409 on Saturday October 17, 2015 @01:58PM (#10405357)
Thank you for the recommandations! Would love to know more about your exact use cases - we're aiming for the 100% :) If you'd reach out on [email protected] - we'd be happy to investigate if we can't make CloudScrape work for you as well.