Re:Lucene: The Good Parts (2015)
by tomwheeler on Wednesday March 30, 2022 @02:27PM
(#30858949)
About ten years after creating Solr, Yonik Seeley joined Cloudera to work on integration between Apache Hadoop and Solr.
There's an interesting connection here: Doug Cutting, perhaps best known as the creator of Hadoop, was the Chief Architect at Cloudera. Most people recognize Doug as the creator of Apache Hadoop, but he also created Lucene. In fact, Hadoop originated from a Lucene subproject called Nutch, which aimed to build a scalable web crawler.
Wasn't the inspiration for Hadoop not just a web crawler use-case, but also Google's famous MapReduce paper?
Re:Lucene: The Good Parts (2015)
by tomwheeler on Wednesday March 30, 2022 @04:14PM
(#30860229)
Yes and no. The goal of the Nutch project was simply to create a web crawler, but it hit some scalability limits. Since Google had recently published two papers (MapReduce and Google Filesystem) that were quite relevant to scaling data processing and storage for a web crawler, Doug and Mike created an open source implementation of those ideas and redesigned the web crawler to use it.
The technology had many applications beyond a web crawler, of course, but that was the original use case.
Solr is a great tool, we use it at my job to index documents and never had any problem with it. We choosed it over elasticsearch because it seems simpler to setup and administer.
I have no idea how I never knew that CNET created Solr! (Solr uses Lucene)