Re:Bitmap Indexes in Go: Search Speed (2019)
by skingkong on Thursday September 22, 2022 @12:23PM
(#32941148)
100%, and are freaking fast...and pretty efficient, especially for high throughput applications tracking tons of changing data. We've found bitmaps handle updates, inserts, and schema changes really elegantly - and quickly (like sub-second). Depending on the use case, looking at cohorts MoM or YoY that freshness doesn't entirely matter, but computing cohorts in real-time hour-over-hour, or even minute-over-minute, for in-app actions/personalization is insanely valuable to keep users engaged (or what ever entity type you are analyzing and taking action on, whether it's devices, nodes across a network, etc).
Re:Bitmap Indexes in Go: Search Speed (2019)
by JAA1337 on Thursday September 22, 2022 @10:50AM
(#32939598)
I think it's fair to say that the use cases for bitmaps have not yet been fully explored. I believe their use in areas of analytics and OLAP/RTOLAP will continue to grow as the desire to infer and target demo's across households increases.
Re:Bitmap Indexes in Go: Search Speed (2019)
by eurasiantiger on Thursday September 22, 2022 @12:25PM
(#32941170)
Isn’t Lucene already a household name?
Re:Bitmap Indexes in Go: Search Speed (2019)
by gthrone on Thursday September 22, 2022 @12:52PM
(#32941503)
I didn't realize Lucene used their own flavor of roaring bitmaps! Super Cool. I certainly think of search when it comes to Lucene, where as the cited material from Vikram & FeatureBase are targeting more general analytics.
Re:Bitmap Indexes in Go: Search Speed (2019)
by kordlessagain on Thursday September 22, 2022 @01:31PM
(#32942066)
Lucene is fast, but probably can't handle analytical type questions on massive amounts of data as fast as a pure binary index can. I could be wrong about that though. I do know Solr has it because Lucene does...
Re:Bitmap Indexes in Go: Search Speed (2019)
by eurasiantiger on Thursday September 22, 2022 @03:43PM
(#32943728)
Elasticsearch (another household name) is built on Lucene too, so I wouldn’t count it out.
Re:Bitmap Indexes in Go: Search Speed (2019)
by clowen on Thursday September 22, 2022 @12:41PM
(#32941372)
Probably not, first thought I had was Lucerne dairy products.
Re:Bitmap Indexes in Go: Search Speed (2019)
by JAA1337 on Thursday September 22, 2022 @12:45PM
(#32941419)
Not sure I understand your point. Can you clarify some?
Re:Bitmap Indexes in Go: Search Speed (2019)
by eurasiantiger on Thursday September 22, 2022 @03:39PM
(#32943676)
Lucene uses roaring bitmaps for filters.
Re:Bitmap Indexes in Go: Search Speed (2019)
by JAA1337 on Thursday September 22, 2022 @04:19PM
(#32944097)
Sure, I completely get it. However, does that mean bitmaps have been fully explored? For example, OLAP is only growing as a market as the need for realtime analysis on the most fresh data. I mean, we never going to have 'less' data nor are we ever going to seek results 'slower'. Where I am going with this, utilizing technologies like bitmaps in OLAP services is an area I believe will see continued growth. While Lucene is a known commodity, I think we can agree it hasn't solved the sector, right?
https://vikramoberoi.com/using-bitmaps-to-run-interactive-re...