Re:SQLite small blob storage: 35% Faster Than the Filesystem
by notacoward on Wednesday June 14, 2017 @07:54AM
(#14552051)
If you store them in sqlite, how do you deal with redundancy / failover / scaling? There are other databases with better tooling for this, but they weren't part of this comparison. The world is full of tradeoffs. Scale and availability vs. microbenchmark performance is one of them. It would be very interesting to see a comparison of clustered MySQL or Postgres vs. Gluster or Ceph. I very much doubt the performance side of it would favor the database so much.
Re:SQLite small blob storage: 35% Faster Than the Filesystem
by fb03 on Wednesday June 14, 2017 @08:00AM
(#14552074)
> If you store them on a filesystem, how do you deal with redundancy / failover / scaling ?
In most cases, using a solution like Amazon S3 will have all these three sorted out for you pretty good. And while it's not a full mountable filesystem, from a system development perspective it has a really good API for retrieving and storing files.
Re:SQLite small blob storage: 35% Faster Than the Filesystem
by eric_the_read on Wednesday June 14, 2017 @12:00PM
(#14553644)
Technically there is s3fs, but it's fairly terrible and I actively recommend against it. Still, it does come handy from time to time, in a limited set of cases.
If you store them on a filesystem, how do you deal with redundancy / failover / scaling ? At a previous job we did a bit of experimenting with using a clustered FS but they all introduced a lot of problems. However, this was a couple of years ago so the situation may be different now.