News for hackers. Stuff that matters.
Re:Real-Time Full-Text Search with Luwak and Samza by huskyr on Monday April 13, 2015 @03:40PM (#9369899)
Bit offtopic, but does anyone have suggestions for simple full text search engines? I basically want something for names, just a couple of thousand, nothing fancy. Setting up something like ElasticSearch seems like overkill (and is quite hungry for specs as well). I was thinking about simply hacking something together with Redis and Python, but i suppose someone might have a better solution.
Re:Real-Time Full-Text Search with Luwak and Samza by ignoramous on Monday April 13, 2015 @04:09PM (#9370108)
Sorry if come off as naive (as I don't really understand what a 'full text search' is), but for a couple thousand names wouldn't grep with regex wildcards suffice?
Re:Real-Time Full-Text Search with Luwak and Samza by huskyr on Monday April 13, 2015 @04:14PM (#9370147)
That doesn't sound like a bad suggestion at all. I think it might get bigger over time, or i might have aliases for the names, and in the end using just regex i'll probably hit a ceiling sooner than later.
Re:Real-Time Full-Text Search with Luwak and Samza by YorickPeterse on Monday April 13, 2015 @04:01PM (#9370055)
If you're using PostgreSQL you can take advantage of its full text search support. When doing so, make sure to save the text search vectors in a physical column as otherwise queries will be quite slow.
Re:Real-Time Full-Text Search with Luwak and Samza by huskyr on Monday April 13, 2015 @04:16PM (#9370151)
I would definitely take that option if we would use PostgreSQL, but we don't. PostgreSQL sounds like a wonderful piece of software, but they need to work on making it more accessible and user friendly. I tried getting something up and running, but given that there isn't a proper native client for Mac i gave up :(
Re:Real-Time Full-Text Search with Luwak and Samza by cglace on Monday April 13, 2015 @06:01PM (#9370801)
It is really easy to install using http://brew.sh/
Re:Real-Time Full-Text Search with Luwak and Samza by huskyr on Tuesday April 14, 2015 @03:51AM (#9372709)
The command-line part and executables are indeed, but it's lacking in visual tools. Say, the equivalents of Sequel Pro and Phpmyadmin.
Re:Real-Time Full-Text Search with Luwak and Samza by ewams on Tuesday April 14, 2015 @10:26AM (#9374476)
Re:Real-Time Full-Text Search with Luwak and Samza by richardbrevig on Tuesday April 14, 2015 @03:07AM (#9372554)
What exactly are you trying to do?

ElasticSearch isn't that difficult and the new guide they published a few months ago walks you through getting it set up without going too in-depth [1]. As a beginner I had it running on a $5 digitalocean droplet within a couple hours and indexed way over "a couple thousand" documents before the end of the day.

But if your needs are really that simple, MySQL does support full-text search [2].

[1] http://www.elastic.co/guide/en/elasticsearch/guide/current/i... [2] https://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html

Re:Real-Time Full-Text Search with Luwak and Samza by huskyr on Tuesday April 14, 2015 @03:54AM (#9372718)
Yes, i've looked into ES before, and if my usecase would be anything more complicated i would definitely go for that route. However, it's still quite some work to setup, and they recommend at least 8GB of RAM for a working instance [1].

I didn't know MySQL had fulltext search as well. I'll look into that. Thanks.

[1]: http://www.elastic.co/guide/en/elasticsearch/guide/master/ha...

Re:Real-Time Full-Text Search with Luwak and Samza by mtrn on Monday April 13, 2015 @07:52PM (#9371266)
I've worked a bit with a nice search library in Go called bleve[1]. That said, it is a library and you would have to implement a server component yourself.

Bleve comes with a few command line utils: https://github.com/blevesearch/bleve/tree/master/utils

[1] http://www.blevesearch.com/

Re:Real-Time Full-Text Search with Luwak and Samza by huskyr on Tuesday April 14, 2015 @03:54AM (#9372719)
Thanks!
Re:Real-Time Full-Text Search with Luwak and Samza by ddorian43 on Monday April 13, 2015 @03:54PM (#9370005)
Python has a search-engine: whoosh
Re:Real-Time Full-Text Search with Luwak and Samza by huskyr on Monday April 13, 2015 @03:58PM (#9370037)
Whoosh sounds pretty much what i'm looking for. Thanks!
Re:Real-Time Full-Text Search with Luwak and Samza by mhuffman on Tuesday April 14, 2015 @02:45PM (#9376409)
Also, if you are using Python, sqlite3 has some pretty advanced text searching capabilities that will scale past your "few thousand" with no problem.