News for hackers. Stuff that matters.
Re:Hyperdrive v10 – a peer-to-peer filesystem by imhoguy on Thursday May 14, 2020 @03:06PM (#23183940)
Does it support private clustering and encryption? My use case is to have shared volumes between servers, laptops but without all the networking hassle (VPNs, opening firewalls).
Re:Hyperdrive v10 – a peer-to-peer filesystem by hinkley on Thursday May 14, 2020 @03:21PM (#23184127)
My go-to example for 'private cloud' (which as soon as we hit the Trough of Disillusionment on this Hype Train, we will all be talking about) is having a set of people in your social or family group maintaining a file share for your semi-private photos and videos.

You don't have to worry about a flood or tornado taking out all the photos of first birthdays, or Grandma when she was younger than you are now, and you also don't broadcast family dynamics, whereabouts (people have been robbed when social media made it clear they were not at home) or pictures of minors.

You, your uncle who Moved to the City and your cousin by marriage who wants to be a game designer all set up a file server and share the photos (your cousin is gonna throttle traffic while he's playing CoD or Dota of course, which it turns out he does all the time but at least he's an extra backup copy).

I don't know how you keep that one relative from uploading funny things they found on the internet that keep trying to install spyware or back doors though, but I suppose you'd have that problem now.

Re:Hyperdrive v10 – a peer-to-peer filesystem by beckingz on Thursday May 14, 2020 @03:33PM (#23184283)
Do you have any good solutions for this exact use case yet?
Re:Hyperdrive v10 – a peer-to-peer filesystem by imhoguy on Thursday May 14, 2020 @03:59PM (#23184598)
Checked Tahoe-LAFS, Perkeep, but they are too low level. Haven't tried Resilio Connect yet, although I would prefer something open-sourced.

I think we have all the tech needed already (WebRTC, DHT, NAT hole punching, decades of p2p, encryption, onion security, StorJ/Filecoin) etc but what is lacking is dead simple UX and wide support of operating systems - Windows, Mac, Android, iOS, Linux (Raspberry, Synology, cheap VPS backup)

Re:Hyperdrive v10 – a peer-to-peer filesystem by hinkley on Thursday May 14, 2020 @04:27PM (#23184934)
Yeah there are no turnkey solutions that I'm aware of. Which is why every time there's a thread like this I come to see what the rumpus is.

I'm still waiting for a 'Drobo' like device without the proprietary physical layout. A light or dial goes into orange territory, you head to Best Buy or Amazon and buy the biggest drive that doesn't give you sticker shock, you push a button, out pops your worst drive and in goes the new one. Some lights flicker for a while and then go green.

I thought ZFS would have given us almost everything but the hardware ten years ago, but it turned out they oversold a few of the features back then, and then Larry happened.

Custom hardware is too expensive for small run consumer hardware, and Apple might have gotten into that space but never did. I wonder how many PCIe lanes you could shoehorn onto a Pi clone...

Re:Hyperdrive v10 – a peer-to-peer filesystem by mceachen on Thursday May 14, 2020 @08:47PM (#23187599)
Synology hardware is pretty close to what you're describing. Light on front goes from green to orange and you get an email, you plug in another drive, click a button in the GUI, and wait for the volume to reshard/resilver/remirror.

A freenas box does this as well, but won't have the pretty drive light indicator if it's a home-built box, but then you're not limited to proprietary hardware.

Re:Hyperdrive v10 – a peer-to-peer filesystem by hinkley on Friday May 15, 2020 @01:12AM (#23189105)
I had been meaning to look into Synology more and I watched a few reviews after this exchange. Sound good except I'm not happy about having to link to their servers. But everything works that way these days :/
Re:Hyperdrive v10 – a peer-to-peer filesystem by thijsvandien on Friday May 15, 2020 @07:40AM (#23191005)
Why would you have to? Their devices work fine without any sort of account with them.
Re:Hyperdrive v10 – a peer-to-peer filesystem by Reelin on Friday May 15, 2020 @12:11AM (#23188824)
4 TB drives are now commonly well under $100 (I've seen them as low as $70) and many Micro-ATX and NUC boards have 4+ SATA connectors. Btrfs has CoW, subvolumes, snapshots, and data integrity features.

For the physical device at least, Debian and 2 Btrfs data drives in RAID 1 certainly isn't turnkey but seems quite accessible at this point.

Re:Hyperdrive v10 – a peer-to-peer filesystem by tekknolagi on Thursday May 14, 2020 @07:44PM (#23187142)
Syncthing works pretty well here!
Re:Hyperdrive v10 – a peer-to-peer filesystem by newman314 on Thursday May 14, 2020 @04:09PM (#23184721)
I use Resilio Sync to sync my 1Password. It mostly works but is clunky in that the handling of identities is clumsy (unable to cleanly remove) as well as having less than stellar UX for adding/removing folders.

Still looking for the perfect Dropbox-like experience but without the cloud sync piece. If only syncthing had decent mobile apps...

Re:Hyperdrive v10 – a peer-to-peer filesystem by Reelin on Thursday May 14, 2020 @11:54PM (#23188738)
Perhaps I'm completely misunderstanding something but don't centralized self hosted cloud-plus-app services address this exact usecase? Things like NextCloud, Seafile, and Sandstorm?

The only major drawback seems to be that you have to host the physical hardware yourself due to lack of solid end-to-end encryption for most platforms. Sandstorm might have it (I'm not clear on what's client- and what's server- side there), Seafile has end-to-end encryption that doesn't protect metadata (https://forum.seafile.com/t/how-strong-is-the-encryption/627...), and NextCloud appears to have a long-running beta of end-to-end encryption on a per-folder basis (https://nextcloud.com/endtoend/). Apparently Cryptomator exists (https://cryptomator.org/) although I've never tried it myself.

If you want to decentralize your shared files unfortunately last I checked SyncThing didn't yet support end-to-end encryption. (I don't think any of the other ones I mentioned can be used in a decentralized manner but things move quickly so I'm not sure.)

Alternatively, if you were thinking more chat and messaging there's self hostable federated services such as PixelFed, PeerTube, Mastodon, and Matrix. Or was there some other usecase you had in mind?

Re:Hyperdrive v10 – a peer-to-peer filesystem by dpflug on Friday May 15, 2020 @02:32PM (#23195630)
SyncThing does support end-to-end encryption, and has for a while: https://docs.syncthing.net/users/security.html#sync-connecti...
Re:Hyperdrive v10 – a peer-to-peer filesystem by Reelin on Friday May 15, 2020 @10:54PM (#23200070)
By end-to-end, I meant that the server can't see your data. For example, the idea of deploying one or more SyncThing instances to cloud providers that would never see your unencrypted data. There's an open GitHub issue about it since 2014 (https://github.com/syncthing/syncthing/issues/109).
Re:Hyperdrive v10 – a peer-to-peer filesystem by mceachen on Thursday May 14, 2020 @04:23PM (#23184890)
I'm hoping PhotoStructure solves this usecase when I add sharing in the next version. Note that PhotoStructure is focused on just photos and videos--it isn't a general file sharing solution.

I'm looking into integrating with DAT or Hyperdrive-like solutions to help make storage backups less fiddly for users.

For now, I recommend my beta users use SyncThing or Resilio Sync to get their photos and videos off their phones and on to their home NAS or computer.

Re:Hyperdrive v10 – a peer-to-peer filesystem by mafintosh on Thursday May 14, 2020 @03:14PM (#23184032)
It uses Noise to encrypt all transport connections and using the api you can whitelist which peers (by their public key) you want to replicate with. This API is not yet exposed through the daemon but it’s in the modules.

Hyperdrives also have a built in capability system where you have to know the public key of the drive to download it from a peer, so if you only share the key with yourself no one else can access it.

Finally using the modules you can build almost any kind of networking you’d imagine but that part requires more work on your side obviously.