News for hackers. Stuff that matters.
Re:Realtime Search: Solr vs Elasticsearch by sbochins on Tuesday May 31, 2011 @12:24PM (#2603200)
What does the author mean by while indexing? You have to send a commit message to make the newly indexed doc searchable. Its silly to do this after every document. I'm assuming that is what the author was doing to get search poor search results. You could also just use Lucene's NRT abilities in Solr w/ a simple patch. I believe it is going to be working out of the box in the next version of solr. I wrote one myself to deal with real time search. If we want a fair comparison, we should be comparing Solr using NRT and ElasticSearch.
Re:Realtime Search: Solr vs Elasticsearch by diego on Tuesday May 31, 2011 @01:00PM (#2603367)
It is not silly if you have asynchronous indexing because your content is generated by your users, one document at a time. If you want real-time indexing, that's the way to do it.

This is the default behavior for IndexTank. When an indexing call returns, whatever you indexed WILL show up in search results. We don't give you the option to commit or not commit, we simply designed IndexTank to be fast enough when indexing in real time. I'd like to see us included in that comparison as well.

Re:Realtime Search: Solr vs Elasticsearch by khill on Tuesday May 31, 2011 @04:40PM (#2604452)
Lucene doesn't have to commit document updates to make them searchable. You just need to grab the IndexReader via the IndexWriter.getReader() method instead of opening an IndexReader on the index directory.

We use this for NRT indexing currently on our site and it works fine - there's a small (500 millisecond or less) delay in changes appearing but it's close enough for us.

We don't use Solr, though - it's a custom search engine based on Lucene.

Re:Realtime Search: Solr vs Elasticsearch by sbochins on Tuesday May 31, 2011 @02:53PM (#2603930)
Its not the only way to do it. Twitter doesn't do it that way, neither does LinkedIn. They have their own Search Engines as well build around Lucene. I haven't looked at them in a while, but I don't think they were commiting every document. They also get their content from their users.
Re:Realtime Search: Solr vs Elasticsearch by Yeroc on Wednesday June 01, 2011 @12:40AM (#2605977)
FYI, LinkedIn open-sourced the work they did to build real-time indexing/search on top of Lucene. It's called Zoie (http://javasoze.github.com/zoie/), I've played with it and it's nice from a performance point of view...
Re:Realtime Search: Solr vs Elasticsearch by diego on Tuesday May 31, 2011 @04:56PM (#2604531)
Correct, Twitter in fact did a lot of work on the Lucene core specifically for this. The OP was talking about standard Solr though.
Re:Realtime Search: Solr vs Elasticsearch by true_religion on Tuesday May 31, 2011 @01:35PM (#2603551)
What's IndexTank using as a backend?
Re:Realtime Search: Solr vs Elasticsearch by diego on Tuesday May 31, 2011 @01:51PM (#2603643)
IndexTank :) Meaning, we wrote a search engine from the ground up. We probably should write a blog post to explain why. We did use a couple of things from Lucene:

- The tokenizers, to be compatible with the query syntax - The index format for persistence on disk. It's a great format, no point in reinventing that. Analogous to how Open Office can save in .doc format.

Re:Realtime Search: Solr vs Elasticsearch by kimchy on Tuesday May 31, 2011 @12:50PM (#2603332)
NRT is just part of the story, since in order to provide a proper (near) real time features in a distributed architecture you need to build it around it. For example, not doing pull replication, but push replication. The whole distributed nature of elasticsearch compared to Solr was not discussed in this blog post, but its a big differentiator.