Re:Zstandard – Real-time data compression algorithm
by killingtime74 on Sunday September 04, 2022 @10:30PM
(#32719857)
The dictionaries mentioned here speed up for small files where the dictionary would be bigger than the raw source. You don't need that for a Mysql dump. Its many times bigger than the dictionary. You could theoretically save space for large files by sharing a dictionary but the saving is probably negligible.
Re:Zstandard – Real-time data compression algorithm
by Cyberdog on Sunday September 04, 2022 @11:00PM
(#32720036)
Okay, interesting. I guess I'll stick with what I've got now for the time being, then. Thanks.
Re:Zstandard – Real-time data compression algorithm
by johannes1234321 on Monday September 05, 2022 @06:50AM
(#32722861)
Sorry for the little hijacking: An alternative to using "regular" dumps might be MySQL Shells dump utility. Contrary to mysqldump etc. it doesn't save data in SQL but some CSV-like format. This won't solve the "@gmail.com" case but avoid the repetitive "INSERT INTO tablename VALUES (" for each record and what isn't there doesn't have to be compressed to begin with. (And then is paralizable etc., downside might be that it is not using a single file, but a number of files)
https://dev.mysql.com/doc/mysql-shell/8.0/en/mysql-shell-uti...
Depending on your needs you could also look at the MySQL clone plugin, which gives you a simple way to clone the data dictionary as physical backup, avoiding the textual representation, but has its restriction. https://dev.mysql.com/doc/refman/8.0/en/clone-plugin.html
Disclaimer: I work at the MySQL development team at Oracle
I imagine that xz might be good enough at making its own dictionary that what it ends up with wouldn't be much different from what I'd make manually, so I could see a decent chance that the improvements would be minimal at best. Has anyone done any experimentation along these lines?