Ubuntu One file syncing code Open Sourced

Re:Ubuntu One file syncing code Open Sourced by rocky1138 on Tuesday August 11, 2015 @11:55AM (#10042003)

Yes. It's very good. I've been using it for about 4 months now between a Raspberry Pi (Raspbian Linux), an old laptop (Ubuntu Linux), my main computer (Windows 8.1), and my girlfriend's computer (Windows 8.1).

[ Parent | on HN ]

Re:Ubuntu One file syncing code Open Sourced by pedrocr on Tuesday August 11, 2015 @12:00PM (#10042046)

It's usable if you have a small amount of files. For anything serious (say 100k files or 500GB of data) is blows through CPU and RAM to the point of being unusable. I've been actively testing several and there is currently no actually good open-source solution in this space.

[ Parent | on HN ]

Re:Ubuntu One file syncing code Open Sourced by rocky1138 on Tuesday August 11, 2015 @12:13PM (#10042159)

I've got about 150GB (but well over 100k files) and it only really lags on the Raspberry Pi. One thing I could suggest is that you change how often it rescans your directories for changes (I set mine to 6000 seconds versus the default 60). That might make a difference.

[ Parent | on HN ]

Re:Ubuntu One file syncing code Open Sourced by pedrocr on Tuesday August 11, 2015 @12:57PM (#10042561)

I've explored this pretty extensively and it's currently just the way it is. It happens from first sync so the rescan interval makes no difference. Your collection probably requires 300-400MB of RAM to sync so it should be on the cusp of failing on a raspberry pi.

Here's the bug report that was closed without any fix:

https://github.com/syncthing/syncthing/issues/468

And user reports of this show up pretty often:

https://github.com/syncthing/syncthing/issues/1841

https://forum.syncthing.net/t/arm-linux-cpu-usage-99-for-man...

https://github.com/syncthing/syncthing/issues/1841

A sync tool that uses on the order 2MB of RAM per GB of files or 4kB per file isn't really usable. A cheap NAS these days can easily have 6TB of RAID6 storage but it won't have 12GB of RAM. Clearly something in syncthing is keeping way too much stuff pinned in memory and the developers so far haven't focused on that. There's no apparent need for this memory usage either as there's already an on-disk database with the state of the sync folder.

[ Parent | on HN ]

Re:Ubuntu One file syncing code Open Sourced by rocky1138 on Tuesday August 11, 2015 @01:38PM (#10042928)

That's really interesting, thanks.

[ Parent | on HN ]

Re:Ubuntu One file syncing code Open Sourced by JeremyNT on Tuesday August 11, 2015 @04:28PM (#10044144)

It uses inotify, so "scanning" for changes more frequently should have only a minimal impact on performance. Of course if you have a lot of actively changing files, you don't want it spending a lot of time transferring files that are still in the process of being modified, so reducing the frequency might help.

I've never noticed performance issues with syncthing, excepting my old original Nexus 7 tablet which does seem to suffer a bit due to its poor hardware. I keep my photos volume (~250GB) synchronized between three desktop systems and it's never presented a problem with those.

Given that the grandparent mentions issues with larger volumes, there may be some sort of "tipping point" beyond which performance degrades, and I just have not encountered it.

Regardless I'd encourage people to give Syncthing a try. The project is under very active development and the issue the grandparent mentions may be resovled by now, and of course that issue might be environment specific and not generally applicable.

[ Parent | on HN ]

Re:Ubuntu One file syncing code Open Sourced by pedrocr on Tuesday August 11, 2015 @07:02PM (#10044980)

> It uses inotify

Actually it doesn't. There are some add-ons to do it but the base tool does periodic full scans.

> I've never noticed performance issues with syncthing, excepting my old original Nexus 7 tablet which does seem to suffer a bit due to its poor hardware. I keep my photos volume (~250GB) synchronized between three desktop systems and it's never presented a problem with those.

250GB between desktops with multiple GB of RAM probably works. It will soak up plenty of resources though.

>Regardless I'd encourage people to give Syncthing a try. The project is under very active development and the issue the grandparent mentions may be resovled by now, and of course that issue might be environment specific and not generally applicable.

The issue is pervasive and apparently quite deeply rooted in the way syncthing is built as it's indeed under very active development but this hasn't improved.

[ Parent | on HN ]

Re:Ubuntu One file syncing code Open Sourced by JeremyNT on Wednesday August 12, 2015 @10:51AM (#10047990)

Thank you for the correction re: inotify, I did not mean to mislead people. Sadly I can no longer edit the incorrect information in my original comment.

[ Parent | on HN ]

Re:Ubuntu One file syncing code Open Sourced by tw04 on Tuesday August 11, 2015 @03:14PM (#10043614)

I second this. Tried to use it to sync a picture directory between my desktop and NAS device and after the original sync, it never managed to complete any further attempts. It's great for smaller directories, it's horrible at lots and lots of small to medium sized files.

[ Parent | on HN ]

Re:Ubuntu One file syncing code Open Sourced by stewbrew on Tuesday August 11, 2015 @12:26PM (#10042300)

Yes. It's easy to install and runs on Android. It's quite fast in detecting differences and then syncing 2 nodes. It's meant to permanently run in the background though. It cannot be not easyly automated to, eg, sync 2 nodes and then shut down. In case if concurrent edits, you end up with multiple versions of the file on disk. You have to somehow detect those conflicts (find) and merge the files yourself. It doesn't help you much with that.

[ Parent | on HN ]

Re:Ubuntu One file syncing code Open Sourced by jhasse on Tuesday August 11, 2015 @11:41AM (#10041867)

Yes! But it's decentralized. An alternative to Ubuntu One would be ownCloud (https://owncloud.org/).

[ Parent | on HN ]

Re:Ubuntu One file syncing code Open Sourced by bildung on Tuesday August 11, 2015 @11:48AM (#10041939)

Another alternative is Seafile https://www.seafile.com/en/download/

[ Parent | on HN ]

Re:Ubuntu One file syncing code Open Sourced by ef4 on Tuesday August 11, 2015 @12:13PM (#10042157)

Centralized is just a degenerate case of decentralized.

Stand up a dedicated server running syncthing and poof!, you have a centralized syncthing.

[ Parent | on HN ]

Re:Ubuntu One file syncing code Open Sourced by baldfat on Tuesday August 11, 2015 @12:26PM (#10042304)

synch Thing is really a good option. It has a good community that is actively supporting the product and the features include only syncing certain files is great. I was a former Bit Torret Sync user but switch due to its open source code.

[ Parent | on HN ]

Re:Ubuntu One file syncing code Open Sourced by mrschwabe on Tuesday August 11, 2015 @11:50AM (#10041957)

And yet another alternative, nodejs based:

http://cozy.io

[ Parent | on HN ]