Re:Why SQLite Is Taking over with Brian Holt and Marco Bambini
by avereveard on Wednesday June 12, 2024 @07:30AM
(#40656856)
SQLite let's you decide when and if vacuuming after partial row modifications. Does arrow allows that I don't know but it's most likely the best SQLite feature you can open a massive database and do one value change really really fast without having to write and compact the entire thing all the times.
[ Parent
| on HN ]
Re:Why SQLite Is Taking over with Brian Holt and Marco Bambini
by andyferris on Wednesday June 12, 2024 @07:42AM
(#40656935)
Arrow isn’t great at random insertion and deletion, as each column is a contiguous array. If it’s on disk those arrays are probably packed tightly together, so you can’t even append rows.
(I have an idea to implement an LSM tree where each layer is an arrow file, which should allow for faster mutations while maintaining a lot of the benefits of arrow. But I haven’t got around to it).