News for hackers. Stuff that matters.
Re:Column – High-performance, columnar, in-memory store with... by kelindar on Monday June 21, 2021 @03:12PM (#27582941)
It's actually possible, columns are simple Go interfaces and can be re-defined and defined for specific types. You can easily build implementation of columns that actually load data from disk or even a remote server (RDBMS, S3, ..?) and retain the indexing capability.

On the flip side, you could actually fit more data in-memory than with non-columnar methods, since the storage is column-by-column, it compresses very well. For example boolean values are stored as bitmaps in this implementation, strings could be stored in a hash map so there's only one string of a type that kept in memory, even if you have millions of rows.

There are no comments on this comment yet.