News for hackers. Stuff that matters.
Re:Whoosh – Fast, full-text indexing and searching library i... by piqufoh on Tuesday November 19, 2013 @07:27AM (#6760653)
Whoosh is great in that it fits the django philosophy of building sites fast, but I wouldn't use if for anything harder than a small site search. Once my site grew larger than 50 mb of text queries started slowing things down. Indexing (python, single threaded) took a while and the larger the index the slower the queries were returned.

It's probably the best first iteration search app that I've come across, and you can always slip in solr or something with more umph when you need it.

Re:Whoosh – Fast, full-text indexing and searching library i... by nknighthb on Tuesday November 19, 2013 @08:23AM (#6760956)
> Indexing (python, single threaded) took a while

The documentation indicates there is a multiprocessing option. (Also configurable memory limits, if you didn't see that.)

http://whoosh.readthedocs.org/en/latest/batch.html

Re:Whoosh – Fast, full-text indexing and searching library i... by piqufoh on Tuesday November 19, 2013 @08:45AM (#6761077)
Ooh - handy, I hadn't seen that thanks. I'll try it out.
Re:Whoosh – Fast, full-text indexing and searching library i... by WoodenChair on Tuesday November 19, 2013 @07:30AM (#6760662)
Can you clarify why you would not use it on anything harder than a small site search? Without context, the comment is not especially helpful.
Re:Whoosh – Fast, full-text indexing and searching library i... by ra on Tuesday November 19, 2013 @07:59AM (#6760827)
For very small datasets, Whoosh is fine. For anything larger than 'site search' on a website that has no UGC, you're much better off with SOLR (or ES, Xapian or Sphinx - whatever your poison).

When document counts get into the hundreds you see orders of magnitude faster queries with SOLR (etc), not to mention much more sophisticated querying options.

Re:Whoosh – Fast, full-text indexing and searching library i... by mattdeboard on Tuesday November 19, 2013 @07:49AM (#6760771)
> Once my site grew larger than 50 mb of text queries started slowing things down. Indexing (python, single threaded) took a while and the larger the index the slower the queries were returned.

I think this makes it pretty clear. Not sure how much more explanation you need.

Re:Whoosh – Fast, full-text indexing and searching library i... by WoodenChair on Tuesday November 19, 2013 @09:01AM (#6761168)
Wasn't in the original comment... yet another example of why HN should not have edit.
Re:Whoosh – Fast, full-text indexing and searching library i... by hackerboos on Tuesday November 19, 2013 @08:16AM (#6760919)
ElasticSearch with Haystack is the next logical move after outgrowing Whoosh.
Re:Whoosh – Fast, full-text indexing and searching library i... by andrewingram on Tuesday November 19, 2013 @08:27AM (#6760981)
Based on my own experiences, by the time search is an important enough feature to outgrow Whoosh, you've also outgrown Haystack's API.