Re:SQLite small blob storage: 35% Faster Than the Filesystem
by adventured on Wednesday June 14, 2017 @12:18AM
(#14550614)
If you don't mind answering, out of sheer curiosity how was the data stored in the individual file/s?
Re:SQLite small blob storage: 35% Faster Than the Filesystem
by akira2501 on Wednesday June 14, 2017 @01:47AM
(#14550863)
Straight array of structs. The complication was that there were several of them, they were stored in bus-attached MRAM which was accessed directly (e.g. mmap of /dev/ram) by several independent processes.
The idea came from previous generations of the product which generally ran on microcontrollers coupled with custom FPGA designs, so, not entirely unwarranted.. but completely incorrect for a newer product with much greater computing power than their previous devices.
This is the hell we just crawled out of in one of our codebases.. it's not so much the access for a single process, but when you want to share the data in the file, that's when it seriously starts to get messy and breaking through otherwise clean abstraction layers and/or forcing recompile on compatibility breaking changes.