News for hackers. Stuff that matters.
Re:Restic 0.13.0 by timmg on Sunday March 27, 2022 @03:58PM (#30823512)
I have a hand-coded backup system for my photo library that writes to S3. It runs every night at 2AM.

The one feature I have that's important to me is this: it will figure out what files need to be uploaded and then upload as many as possible for an hour then stop.

That means that it runs for at-most an hour a night.

The reason I need/wanted this feature is that I might come home from a trip with (eg) 30G worth of photos. My (cable) internet will upload at around 1G an hour. I don't want this thing to saturate my internet for 30 hours straight. Instead, it backs up a small amount every night for 30 days.

Am I the only one that wants a feature like this? I've never seen it in any other backup system. (At alternative might be to have configurable bandwidth for uploads.)

Re:Restic 0.13.0 by lykron on Sunday March 27, 2022 @04:07PM (#30823593)
You could do this with the `timeout` command.

`timeout -s SIGINT 1h restic...`

That would let restic run for one hour, and then once the hour elapses send a SIGINT which will stop the process (see https://github.com/restic/restic/blob/a29777f46794ea4e35548f...)

Re:Restic 0.13.0 by sydney6 on Sunday March 27, 2022 @04:35PM (#30823844)
And restic would then resume the backup next time it runs.

[1] https://restic.readthedocs.io/en/latest/faq.html#will-restic...

Re:Restic 0.13.0 by lucgommans on Sunday March 27, 2022 @04:06PM (#30823580)
Makes perfect sense. Restic kind-of supports this because you can just kill the client after an hour and, tomorrow, it'll see which objects are there already.

I'm not deep enough into the project to know whether this is like an officially supported use-case, but restic was of course made with the idea that interruptions can happen (your computer can crash) and should be handled safely, and for the deduplication it'll cut files up in a deterministic way and thus (as I understand it) store those chunks in a deterministic place.

Re:Restic 0.13.0 by seized on Sunday March 27, 2022 @04:19PM (#30823696)
Rclone will do exactly what you want, upload to S3 and the --max-duration will stop new transfer from starting after a given duration.

There are also throttle options for bandwidth. I use that combined with Node-Red and a smart plug on my monitors, if monitor power draw exceeds a threshold then the upload throttle is changed via the rclone API.

Re:Restic 0.13.0 by timmg on Sunday March 27, 2022 @04:49PM (#30823982)
Ok, that looks like a nice tool. Thanks!
Re:Restic 0.13.0 by n8henrie on Monday March 28, 2022 @01:09AM (#30827325)
Was just going to suggest rclone as well, but for its easily toggle-able bandwidth limiter. I have slow rural internet; restic backs up locally every night, rclone then syncs it offsite. A systemd timer throttles it during walking hours and lets all 6 mbps rip overnight or when out of town.
Re:Restic 0.13.0 by aborsy on Monday March 28, 2022 @02:06AM (#30827578)
Also, Restic has rclone backend!
Re:Restic 0.13.0 by teaudel on Sunday March 27, 2022 @04:12PM (#30823641)
My internet upload speed is bad so I do want something like that.

I would also like to be able to "stage" a backup: figure out what needs to be transmitted and then create the data files that need to be transmitted without actually immediately transmitting it.

That would allow me to do things like backup my laptop to another computer in my house that can upload the files over my slow connection overnight when my laptop isn't on; and to let me bring the backup files to a place (work/university/library) with a fast connection so large backups don't take days or weeks (especially initial backup).

Re:Restic 0.13.0 by pdimitar on Sunday March 27, 2022 @05:47PM (#30824478)
Take a look at NNCP[0]. Haven't tried it yet but read several articles about it and I am looking forward to finding time to use it seriously.

[0] https://www.complete.org/nncp/

Re:Restic 0.13.0 by funcDropShadow on Sunday March 27, 2022 @06:41PM (#30824881)
I am using restic to backup my laptop and workstation to my nas. At night rclone syncs the restic repositories to S3. I can restore both from my nas as well as from s3.
Re:Restic 0.13.0 by eliaspro on Monday March 28, 2022 @03:11PM (#30834369)
On Linux, this can be easily achieved by using RuntimeMaxSec= [1] in the corresponding service unit.

[1] https://www.freedesktop.org/software/systemd/man/systemd.ser...

Re:Restic 0.13.0 by crecker on Sunday March 27, 2022 @04:06PM (#30823579)
I wonder if you can obtain that behavior with a bash script, but it's boring to write scripts and I do not know if SIGTERM can exit Restic gracely.
Re:Restic 0.13.0 by lucgommans on Sunday March 27, 2022 @04:07PM (#30823588)
If you've ever ctrl+c'd restic, you'll know the message

> signal interrupt received, cleaning up

Re:Restic 0.13.0 by crecker on Sunday March 27, 2022 @04:08PM (#30823601)
Thanks! Will try tonight to play with Restic
Re:Restic 0.13.0 by sydney6 on Sunday March 27, 2022 @04:39PM (#30823879)
Restic has support for rate limiting built-in.