News for hackers. Stuff that matters.
Re:Riot – Full-text search engine in Go by Jemaclus on Monday October 23, 2017 @01:01PM (#15534687)
Also, for consideration: Bleve (https://github.com/blevesearch/bleve)

I'm in the process of building my own search engine (as a learning exercise, but also because it's related to my day job). I've learned that it's one thing to write a full-text search engine, like this one, and it's quite another to do field-specific searches with faceting support and so on, like Algolia and Lucene-based search engines do.

That said, this is clean and simple. I like it. I can definitely learn from this.

Re:Riot – Full-text search engine in Go by markpapadakis on Monday October 23, 2017 @01:34PM (#15535039)
Supporting faced search and other functionality requiring access to per document field-values is just an extension over the core IR functionality.

Tracking (document, field) values can be used for query by range or by geolocation primitives (that's what Lucene does, where it will index that data into a special tree-like structure, and for each query, it will build a custom 'iterator' and use it along with other iterators to match documents), and for static ranking of matched documents.

BTW, Lucene and Algolia are vastly different in terms of the underlying architecture.

Re:Riot – Full-text search engine in Go by lobster_johnson on Monday October 23, 2017 @01:52PM (#15535282)
What is Algolia's underlying architecture like? Are there any papers or code?
Re:Riot – Full-text search engine in Go by markpapadakis on Monday October 23, 2017 @01:54PM (#15535293)
See links about Algolia arch (and other related material) here: https://github.com/phaistos-networks/Trinity/wiki/IR-Search-...
Re:Riot – Full-text search engine in Go by gdillon on Monday October 23, 2017 @04:13PM (#15536562)
Thanks for the links, Mark! (Disclaimer, I work for Algolia.)

As Mark mentions on his summary page, the best place for that kind of information is our CTO's "Inside the Engine" series (8 parts).

https://blog.algolia.com/inside-the-algolia-engine-part-1-in...

Re:Riot – Full-text search engine in Go by Jemaclus on Monday October 23, 2017 @04:29PM (#15536696)
That's why I mentioned them separately. :)
Re:Riot – Full-text search engine in Go by amelius on Monday October 23, 2017 @04:33PM (#15536730)
Could you please include a bibliography in your project? That way, others can more easily find which techniques you are using, and of course it can help others (or even your future self) in figuring out how the code works.
Re:Riot – Full-text search engine in Go by CSDude on Monday October 23, 2017 @01:14PM (#15534803)
Last time I checked, bleeve was very slow compared to Lucene, I really hope it gets better over time.
Re:Riot – Full-text search engine in Go by StreamBright on Monday October 23, 2017 @01:55PM (#15535306)
I was planning to write one too, wanna team up? :)