News for hackers. Stuff that matters.
Re:Tarsnap outage (July 2-3, 2023) postmortem by mattbee on Thursday July 27, 2023 @03:40AM (#36890228)
The deduping in restic is just on the edge of acceptable for me, making me think I'd have trouble with a lot more data. Basically the one a month "prune" operation takes about 36h (to B2) . I feel I could be tuning something but also it works and I don't want to touch it.
Re:Tarsnap outage (July 2-3, 2023) postmortem by sandgiant on Thursday July 27, 2023 @03:43AM (#36890254)
Curious how much you backup, which version of restic you're running and why you think the deduplication is borderline unacceptable. There were several major (orders of magnitudes) improvements made to pruning within the past ~1 year, that's why I'm interested.
Re:Tarsnap outage (July 2-3, 2023) postmortem by mattbee on Thursday July 27, 2023 @04:13AM (#36890478)
A straight upgrade, that I can do :) It's been running for years without one.

I was only edgy about it because when it takes 36h it blocks the next daily backup, and I wondered whether that was going to get worse (it hasn't).

Re:Tarsnap outage (July 2-3, 2023) postmortem by chibea on Sunday July 30, 2023 @08:57AM (#36930712)
The max-unused percentage feature is well worth it to 80/20 the prune process and only prune the data which is easiest to prune away (i.e. not try to remove small files big packs but focus on packs which have lots of garbage).

In general, there's an unavoidable trade-off between creating many small packs (harder on metadata throughout the system, inside restic and on the backing store but more efficient to prune) versus creating big packs which are more easy on the metadata but might create big repack cost.

I guess a bit more intelligent repacking could avoid some of that cost by packing stuff together that might be more likely to get pruned together.

Re:Tarsnap outage (July 2-3, 2023) postmortem by aborsy on Thursday July 27, 2023 @04:30AM (#36890596)
I backup around 2TB with Restic, also tried locally with Borg. The size is nearly the same. Sadly, I can’t even test with Tarsnap! (absurd pricing for 2TB).
Re:Tarsnap outage (July 2-3, 2023) postmortem by justsomehnguy on Thursday July 27, 2023 @05:25AM (#36891011)
> absurd pricing for 2TB

Well, it can't be that ba..

    $0.25 x 2000 = $500
Yikes. And this is without BW costs.

At $500/M you can just rent a dedicated physical server with a lot of HDDs and still have money left for your favourite pumpkin latte.

For comparison rsync.net says it's $0.015 per GB/Mo, for 2TBs that's $30/m and no BW costs.

Re:Tarsnap outage (July 2-3, 2023) postmortem by jiripospisil on Thursday July 27, 2023 @07:05AM (#36891804)
Not in any way affiliated but I'm a happy user of Scaleway's Object Storage [0] together with S3QL [1]. It's not the fastest but they give you 75GB of storage for free so that's a fair trade [2].

[0] https://www.scaleway.com/en/object-storage

[1] https://github.com/s3ql/s3ql

[2] https://www.scaleway.com/en/pricing/?tags=storage

Re:Tarsnap outage (July 2-3, 2023) postmortem by ilyt on Thursday July 27, 2023 @08:49AM (#36892637)
I'm renting $15/mo 2TB atom machine from OVH/kimsufi as second target for backups.

Now that I think about it... some kind of micro-distributed backup server (throw on few of your machines, auto-replicate between) would be a neat project...

Re:Tarsnap outage (July 2-3, 2023) postmortem by justsomehnguy on Thursday July 27, 2023 @09:45AM (#36893386)
It's not even that neat.

Just slap rsync/syncthing to the backup dir.

Re:Tarsnap outage (July 2-3, 2023) postmortem by ilyt on Friday July 28, 2023 @10:55AM (#36907680)
I do use syncthing on NAS + remote cheapo server for my day to day stuff, and bareos for rest.

It's just PITA to add another instance.