gioark
Re:Making the Internet Archive’s full text search faster
on Saturday June 10, 2017 @11:28AM (#14527680)
sure, feel free to contact me by email or DM on twitter.
gio archive org
Re:Making the Internet Archive’s full text search faster
on Saturday June 10, 2017 @11:25AM (#14527668)
We don't use stopwords because we want to find all the best and complete matches. We don't want to ignore any of the words part of the search query.
Re:Making the Internet Archive’s full text search faster
on Friday June 09, 2017 @05:44PM (#14524839)
hi, I am the author of the article.
@devhead: To import the data into ES we used a custom application to extract the text from the OCR'd documents. This is required to support our bookreader software. A complete ingestion takes a few days; we rate-limit indexing in order not to overload the cluster, and maintain reasonable search performance.
Re:Making the Internet Archive’s full text search faster
on Friday June 09, 2017 @04:52PM (#14524453)
If you are having problems loading the page, try this url:
https://medium.com/@giovannidamiola/making-the-internet-arch...