Re:Bleve: full-text search and indexing for Go
by wejick on Saturday January 06, 2018 @10:59PM
(#16089060)
Yes and if I'm not mistaken that's what alias on bleve for.
Not yet looking into scorch. So scorch would replace other storage engine like rocksdb and leveldb?
Re:Bleve: full-text search and indexing for Go
by mschoch on Saturday January 06, 2018 @11:06PM
(#16089082)
Yes, when you create a new index, you can choose the index implementation. The previous one was called 'upsidedown', and this one used key/value stores for the actual storage. So, you would also specify BoltDB/RocksDB/LevelDB/Moss/etc. The new index implementation is called 'scorch' and it writes directly to disk instead of going through a key/value store.
Merging is required and is indeed somewhat resource intensive. Bleve's current indexing approach has no segments, instead all index data is serialized into a key/value store. This approach allowed us to experiment and plug-in a variety of implementations. Unfortunately, the key/value abstraction limits the way you interact with data, so there are a number of drawbacks. One key gain we get from the segmented approach vs the key/value store approach is that we no longer need to maintain a backindex to handle updates/deletes.