Re:Web search 30M rows sub-second, cheaply
by fizx on Sunday November 22, 2015 @07:00PM
(#10612089)
Did you benchmark against GIST? Its hard to tell because you posted very loose benchmarks on a shared VM, but the performance wasn't particularly impressive.
Is there a reason that you're unwilling to trust Postgres's query planner/buffer cache to keep your index in RAM? Even if it didn't, the premise of the article is that SSDs are fast. You're spending an order of magnitude more time retrieving content than searching for it, so why optimize the fast part?
Also, consider that GIST or another fulltext aware index (e.g. Lucene/Sphinx/etc) should compress your index (with vints, etc) far better than your approach, and therefore both query faster and save you precious RAM.
The article itself was sparse on details, but until I learn more, this seems to me to be a classic example of optimizing before measuring.
Is there a reason that you're unwilling to trust Postgres's query planner/buffer cache to keep your index in RAM? Even if it didn't, the premise of the article is that SSDs are fast. You're spending an order of magnitude more time retrieving content than searching for it, so why optimize the fast part?
Also, consider that GIST or another fulltext aware index (e.g. Lucene/Sphinx/etc) should compress your index (with vints, etc) far better than your approach, and therefore both query faster and save you precious RAM.
The article itself was sparse on details, but until I learn more, this seems to me to be a classic example of optimizing before measuring.