Re:Why Polars rewrote its Arrow string data type
by rvilj on Thursday August 08, 2024 @11:40AM
(#41192717)
The length of the buffer for entire column in O(n * k). If you want to gather a subset of the strings in that buffer, you essentially need to scan across the buffer and copy out the subset you want. Because that buffer grows with string size k, it's linear wrt k. The new implementation doesn't have this problem - you only need to deal with views which are constant size.
Can anyone explain why this is?