Internet Archive breached again through stolen access tokens

Re:Internet Archive breached again through stolen access tokens by ChadNauseam on Sunday October 20, 2024 @02:46PM (#41897571)

I wish IPFS wasn't so wasteful with respect to storage. I tried pinning a 200mb PDF on IPFS and doing so ended up taking almost a gigabyte of disk space altogether. It's also relatively slow. However its implementation of global deduplication is super cool – it means that I can host 5 pages and you can host 50, and any overlap between them means we can both help one another keep them available even if we don't know about one another beforehand.

For a large-scale archival project, it might not be ideal. Maybe something based on erasure coding would be better. Do you know how LOCKSS compares?

[ Parent | on HN ]

Re:Internet Archive breached again through stolen access tokens by diggan on Sunday October 20, 2024 @05:17PM (#41898547)

> I tried pinning a 200mb PDF on IPFS and doing so ended up taking almost a gigabyte of disk space altogether

Was that any file in particular? I just tried it myself with a 257mb PDF (as reported by `ls -lrth`) and doesn't seem to add that much overhead:

    $ du -sh ~/.ipfs
    84K     /home/user/.ipfs

    $ ipfs add ~/Downloads/large\ PDF\ File.pdf
    added QmSvbEgCuRNZpkKyQm6nA5vz5RTHW1nxb6MJdR4cZUrnDj large PDF File.pdf
     256.58 MiB / 256.58 MiB [============] 100.00%

    $ du -sh ~/.ipfs
    264M    /home/user/.ipfs

[ Parent | on HN ]