News for hackers. Stuff that matters.
Re:Mistakes Beginners Make When Working with Databases by PaulHoule on Wednesday June 08, 2016 @12:23PM (#11863367)
Years ago I built an image server that served up millions of tiny images (1k or so) and found mysql did a pretty good job for that, avoiding the file system overhead which would overwise be awful.
Re:Mistakes Beginners Make When Working with Databases by rwmj on Wednesday June 08, 2016 @12:51PM (#11863614)
I think it very much depends on specifics. Years and years ago I stored emails as BLOBs in an Informix database for a large webmail system. That turned out to be a bad idea and we ended up migrating (very painfully) millions of emails to a file server.

However I don't think it's universally true this would always be bad. It depends on the speed at which you can insert and fetch blobs from your database, the speed of your NFS server, whether your database or file server can be replicated, and the problems with data integrity you're definitely going to have when not everything is known by the database.

No substitute for careful analysis of individual cases.

Re:Mistakes Beginners Make When Working with Databases by exceptione on Wednesday June 08, 2016 @01:04PM (#11863731)
How did MySQL help here? It uses the filesystem too...

I can imagine that you want to hold these images in RAM to bypass the filesystem. Maybe MySQl did that for you, but that sounds like a heavy middleman.

Re:Mistakes Beginners Make When Working with Databases by PeterisP on Wednesday June 08, 2016 @01:45PM (#11864048)
"bypassing the filesystem" in this case means not having a million of tiny files, which often hurts performance.

If you'd have non-DB system for that (e.g. any game with a lot of tiny images), then you would generally want to use spritesheets, archives or some other approach to store many of these images in a single file instead of having each of them be a separate entry in the filesystem.

Re:Mistakes Beginners Make When Working with Databases by teraflop on Wednesday June 08, 2016 @03:12PM (#11864778)
Typically, filesystems are designed and tuned around the assumption that most of your data is stored in relatively large files. For instance, a typical ext4 filesystem uses 256 bytes per inode and rounds each data block up to a multiple of 4096 bytes.

So if you want to store lots and lots of tiny objects, it's very inefficient to store them each in a separate file. At that point, you could design your own compound data format, making sure it efficiently supports all the different operations you might want to do... or you could just use a database which has already solved the problem for you.

Re:Mistakes Beginners Make When Working with Databases by pmelendez on Wednesday June 08, 2016 @01:24PM (#11863888)
If you have a great amount of tiny images then you are relying in how fast is the filesystem to localize the image. Chances are that depending on the index you are using MySQL could do a better job (this depends on the storage engine but usually is one file for the table and another ones for indexes)