News for hackers. Stuff that matters.
Re:The Web is missing an essential part of infrastructure: a... by profalseidol on Monday April 22, 2019 @02:15AM (#19716920)
> I don’t think web index would ever be free

Yet the company first mentioned does it for free, lol:

https://commoncrawl.org/

I've checked Datastreamer.io for 5 seconds, I don't see any link to their repo. If not "open source" then what does "open" mean?

Re:The Web is missing an essential part of infrastructure: a... by ksahin on Monday April 22, 2019 @03:35AM (#19717189)
Commoncrawl is not a company, it's a non-profit. Open means you can access the data, there is no assumption about the data being free or not.
Re:The Web is missing an essential part of infrastructure: a... by warent on Monday April 22, 2019 @04:01AM (#19717274)
What? It's a nonprofit organization engaging in nonprofit business. Any organization that engages in business is a "company." Common Crawl is a company. Your comment isn't accurate and it doesn't address the parent's comment.