You are looking for Syncany: https://www.syncany.org/

But really, you may not need S3 at all and just sync between your devices with Syncthing: https://syncthing.net/

Shameless plug: Syncthing is in official Debian repositories! https://tracker.debian.org/pkg/syncthing

"The core team of Syncany is on hiatus for an indefinite amount of time. Feel free to do with the code what the license allows and encourages, but please don't expect any maintenance"

Not feasible to use this for critical infrastructure if there is no maintenance happening.

How have you found syncthing perf? When I tried it a while back the perf was horrible. It ate CPU and was pretty slow, even though I was just testing with 2 machines on my local network.

This was improved recently with 14.0.7:

- https://github.com/syncthing/syncthing/releases/tag/v0.14.7

Also, It will improve further with File System Notifications (from what I understand?):

- https://github.com/syncthing/syncthing/pull/2807

It's still slow and use lots of CPU.

Thanks, Syncthing looks like exactly what I was looking for! I'm guessing it needs some external server for coordination? It seems a bit unnecessary to restart after each config chsnge, is there some technical reason for it? Nice job otherwise, will try it out.

Yes, it needs an exernal server.

By default, Syncthing is pre-configured with community-hosted discovery servers:

- https://github.com/syncthing/syncthing/blob/master/lib/confi...

The community also hosts relay servers, so if your two devices can't communicate with eachother directly, it will work anyway.

Relay servers take bandwidth. Anyone can run a relay server, and it will automatically join the relay pool and be available to Syncthing users. This is documented here:

- https://docs.syncthing.net/users/strelaysrv.html#strelaysrv

It would also be possible to host your own private relay and discovery pools, if you need that for some reason.

I use amazon cloud drive unlimited $60/yr and arq backup to have client side encrypted backups. I also arq backup to my local NAS.

If you want to use an open source tool you can use borg backup & rsync.net as your external backup site. Borg doesn't have good S3 integration, and using fuse & s3 doesn't work that well either. It works best when the borg daemon is on the reciever box too to help with indexing and such.

http://www.rsync.net/products/attic.html

Interesting you mention this ...

There's a big item on one of my whiteboards: "put gmvault into the environment" ... the idea being that you could run 'gmvault', over SSH, on rsync.net:

ssh [email protected] gmvault ... blah blah ...

I've been meaning to do this forever ... it would be great if rsync.net customers could not install anything, but just run gmvault as an ssh command.

The only reason it takes time is that we do not have a python interpreter in our environment - we try to keep things as simple and locked down as possible - which means we have to "freeze" gmvault as a binary executable in order to put it into place ...

So ... folks want this ?

Please don't use OwnCloud. It eats your files and cost us loads of time and effort, in addition to sowing FUD among my office coworkers, who thought someone was deleting files from the shared/sync drive. Plus it doesn't support delta sync [0], so if (for example) you're syncing large files like (for example) True/VeraCrypt volumes, you're going to be pushing a lot of data around. This is especially awful since you're not doing this on a LAN but to S3, which means your raw cost in dollars for operating this software will be much much larger than with another solution like SyncThing or Seafile which does support delta sync.

[0] https://owncloud.org/faq/#partialsyncing

ownCloud, or nowadays NextCloud [1], is still the best solution I have, hosted on a private virtual server. It's not a drop-in replacement for sure -- I'm basically prepared to do a full clean reinstall each time I want to upgrade -- but they have desktop and Android clients, and it's working just fine. Of course, I also have a separate backup of all the files.

[1] https://nextcloud.com

I did an extensive lookup of dropbox alternatives last year, and ended up self-hosting seafile. Multiple family members use it and it works great, plus encryption built-in from day one (me not being able to read their stuff is critical to me). There has been no outage, and upgrades are easy.

You may want to have a look at odrive (https://www.odrive.com). Use it to sync a bunch of different storage accounts via this single app. Works alright for what it does, and they have an S3 option.

How about Tarsnap?

I've been using syncthing and I'm reasonably pleased with it. I don't use it heavily and only over my local network, though, so I'm not sure how well it handles archiving changes and file conflicts.

Won't forklift serve your purpose?