Deduplicating a 10.4 TiB game preservation archive (WIP)

Re:Deduplicating a 10.4 TiB game preservation archive (WIP) by rcxdude on Tuesday December 17, 2024 @07:43PM (#42447057)

Often these archive projects have the goal of propagating the archive across many different systems, for which reducing size is very valuable. This is basically a compression exercise in the case where there are many large duplicated blocks.

[ Parent | on HN ]

Re:Deduplicating a 10.4 TiB game preservation archive (WIP) by brudgers on Tuesday December 17, 2024 @08:42PM (#42447365)

Thanks, that makes sense.

The question began, I am working on a game preservation project.

[ Parent | on HN ]

Re:Deduplicating a 10.4 TiB game preservation archive (WIP) by canucker2016 on Sunday December 22, 2024 @12:30PM (#42487778)

The line you referenced is a statement - not a question.

The third line of the post is "The goals are: - bring the size down - retain good read speed (for further processing/reversing) - easy sharable format - lower end machines can use it"

[ Parent | on HN ]

Re:Deduplicating a 10.4 TiB game preservation archive (WIP) by DrFrugal on Wednesday December 18, 2024 @05:05AM (#42449325)

This is correct - the main goal is to have this rather compact, but still having good read times. This will allow me to store it on my new 4 TiB NVMe drive. A lot of iterative scanning will happen, because I search for interesting information, which helps reverse engineering. Also it allows me to share this with other people over the internet before I kick the bucket in a few decades... transferring 10.4 TiB would be rather boring :D

[ Parent | on HN ]