News for hackers. Stuff that matters.
Re:Mixnode: Turn the web into a database by fooyc on Sunday October 07, 2018 @01:12PM (#18161585)
Is this an alternative to crawling/scraping, or a way to exploit the result of crawling/scraping ?

What they offer is not really clear from the article. It seems that they only provide a raw SQL interface over a database of crawled web pages (to be fair, they added a few HTML-related SQL functions). We don’t know where this database come from, or who is supposed to provide it.

Great to see that SQL is making a come back, though.

Re:Mixnode: Turn the web into a database by testplzignore on Sunday October 07, 2018 @01:58PM (#18161808)
Based on a quick google search (e.g., https://stackoverflow.com/questions/46673751/nutch-vs-heritr...), their existing product appears to be a hosted solution for crawling the web.

This new product sounds like it is just a query language that can be used on top of what you yourself have paid them to crawl. I don't believe they've actually crawled the whole web and are providing an interface to that. Their website says things like "the entire web" and "trillions of rows", but I'm guessing that's only true if you pay them a few million dollars to do that.

Re:Mixnode: Turn the web into a database by ajawee on Sunday October 07, 2018 @11:10PM (#18163888)
I guess, there are using common crawl as a base. Not sure wether they are doing actual crawling along with it.