Re:Tape still beats SSDs and hard drives when it comes to pr...
by adrian_b on Wednesday September 25, 2019 @02:51PM
(#21073926)
As Dylan16807 also said, you waste very little. I am using 6 TB LTO-7 tapes and I group the data written to the tapes in files of approximately 50 GB, which are read from a NVMe SSD.
This means that each 50 GB file is written at 3 Gb/s, the maximum tape speed, but after each 50 GB the tape is stopped then restarted. So I write on each tape about 120 files with an aggregate size of up to 45 905 860 blocks of 131 072 bytes each. That is more than the 6.0 TB advertised for the tapes, so I do not waste any measurable tape capacity. The 50 GB file size is chosen because it fits inside the 64 GB RAM of the computer with the tape drive and certain algorithms that I am using to process the tape files (lrzip & par2, for compression & redundancy) are faster when the entire file fits in the RAM.
This means that each 50 GB file is written at 3 Gb/s, the maximum tape speed, but after each 50 GB the tape is stopped then restarted. So I write on each tape about 120 files with an aggregate size of up to 45 905 860 blocks of 131 072 bytes each. That is more than the 6.0 TB advertised for the tapes, so I do not waste any measurable tape capacity. The 50 GB file size is chosen because it fits inside the 64 GB RAM of the computer with the tape drive and certain algorithms that I am using to process the tape files (lrzip & par2, for compression & redundancy) are faster when the entire file fits in the RAM.