News for hackers. Stuff that matters.
Re:Ask HN: What is your favorite method of sending large files? by arsome on Wednesday September 02, 2020 @06:55PM (#24359161)
Just be careful - unless you mark that torrent private it'll get posted to the DHT and crawlers like BtDig will pick it up and list it publicly.

For this reason I prefer using something like Syncthing which is designed more with this purpose in mind.

Re:Ask HN: What is your favorite method of sending large files? by jivank on Wednesday September 02, 2020 @10:25PM (#24360293)
I am working on a tool that will help with this. It is built with AlpineJS, Nim, Aria2 and Webview resulting in a 5MB download which doesn't include the torrent file as that varies. The idea is that it is a one click solution as the torrent is embedded with the binary.

The inspiration for this tool is for assist with LAN parties. The one thing I have against the private flag is that it also disables LAN peer discovery, which would be okay if you use a tracker behind the LAN (though the torrent would need to be modified if the hostname/IP changes). Since you can't configure other people's clients I found it simpler to use Aria2 that is preconfigured to disable PEX and DHT.

Combine this with Metalink or Web Seeds, so that you can have a initial seeder based on HTTP. I think using IPFS would be a great web seed as long as the their gateway and Cloudfare's continues to stay up. IPFS creates a permanent URL if you will, so no need to worry about dynamic IPs or domains. It would be great if IPFS would have a smaller binary (right now it is around 20MB compressed) and had a way to get a file and "seed" it like a torrent. But for now I think torrent is mature enough.

Re:Ask HN: What is your favorite method of sending large files? by brink on Wednesday September 02, 2020 @10:32PM (#24360334)
Why not use Syncthing or Resilio Sync for LAN parties?
Re:Ask HN: What is your favorite method of sending large files? by jivank on Wednesday September 02, 2020 @10:40PM (#24360362)
I have considered it. The one thing about Syncthing is that there is no true readonly way to sync. For example, someone may accidentally extract an archive in the Syncthing folder and it will sync everywhere. If Syncthing gets this feature I think I would be 100% onboard with it for LAN parties.

Resilio I believe supports it but I would prefer an opensource alternative.

https://github.com/syncthing/syncthing/issues/62

Re:Ask HN: What is your favorite method of sending large files? by csdreamer7 on Wednesday September 02, 2020 @11:59PM (#24360699)
You can use the send only feature in Syncthing to make your folder read only.

https://docs.syncthing.net/users/foldertypes.html#folder-sen...

Re:Ask HN: What is your favorite method of sending large files? by jivank on Thursday September 03, 2020 @01:22PM (#24366324)
Sure, I can have for example my computer be send only. But unless everyone else configures it properly as receive only, they can start pushing data to other clients. It would be far easier to have Syncthing create a readonly share similar to Librevault and Resilio.
Re:Ask HN: What is your favorite method of sending large files? by Polylactic_acid on Wednesday September 02, 2020 @07:26PM (#24359365)
You could also gpg encrypt the file first which is a little more work but bulletproof. Maybe someone should make a tool that creates a torrent and encrypts it for you and presents a password box on the other end.
Re:Ask HN: What is your favorite method of sending large files? by iaml on Wednesday September 02, 2020 @07:38PM (#24359452)
Like magic warmhole?
Re:Ask HN: What is your favorite method of sending large files? by edjw on Thursday September 03, 2020 @04:16AM (#24361866)
Re:Ask HN: What is your favorite method of sending large files? by orionblastar on Wednesday September 02, 2020 @08:35PM (#24359751)
Sort of like Limewire sharing your documents directory?

They have MuWire now: https://muwire.com/

Not private but public sharing.

Re:Ask HN: What is your favorite method of sending large files? by bserge on Thursday September 03, 2020 @03:13AM (#24361523)
That's why you encrypt it or at least make it a password protected archive. That way no one will see the file names in the content list. And without seeders, it's highly unlikely anyone will download it. Torrents work well with 1 seeder, 1 downloader.
Re:Ask HN: What is your favorite method of sending large files? by cgb223 on Thursday September 03, 2020 @12:53AM (#24360895)
Technically speaking, what occurs when you mark a torrent private?

Does it actually block trackers from picking it up or just like tell them not to if they want to obey it?

Re:Ask HN: What is your favorite method of sending large files? by arsome on Thursday September 03, 2020 @01:13AM (#24360972)
I'm not an expert on this, but as far as I'm aware, the main difference is that your client won't go out and announce the infohash to the DHT (used for trackerless torrents, magnet links, that kind of thing) and won't provide it to other clients for download. It'll only ever send requests relating to it to the trackers specifically listed within the torrent itself. This means that anyone who wants to download it basically needs to posses the torrent file. Trackers don't really crawl in this way, however, some searching sites like the one I mentioned there do, they'll go out and look at the entire DHT, trying to find entries that they don't know yet and when they find one - go download the torrent and post it to their index.
Re:Ask HN: What is your favorite method of sending large files? by AnonHP on Thursday September 03, 2020 @02:21AM (#24361256)
For some reason I haven’t understood, syncthing would often miss transferring files but mark the entire transfer/sync as complete. No errors in the logs either. I tried it for a few months earlier this year, then gave up and switched to plain rsync in a shell. Based on that sample of one personal experience, I wouldn’t recommend syncthing to anyone.

Note: I was using syncthing for a one way sync all the time, with send-only and receive-only settings on the endpoints.

Re:Ask HN: What is your favorite method of sending large files? by JeremyNT on Thursday September 03, 2020 @08:50AM (#24363497)
I've been using Syncthing extensively for years and never observed this behavior - it always either completes the transfer or fails to complete the transfer (which is obvious in the UI). A silent failure is certainly not typical.

This sounds like a pretty severe bug, so if you still have the systems you were able to produce the behavior on it might be helpful to reproduce it and submit an issue.

Re:Ask HN: What is your favorite method of sending large files? by nonbirithm on Thursday September 03, 2020 @12:22PM (#24365621)
Yeah, I gave up an Syncthing when using it between my Android device and Linux server. Too many hours were lost trying to debug why the server no longer showed as connected on the phone. At some point I went back to emailing myself files, because that actually worked, and I realized that I didn't actually need an entire program with a synchronization algorithm to send a few large files on the occasion.
Re:Ask HN: What is your favorite method of sending large files? by voltagex_ on Wednesday September 02, 2020 @08:02PM (#24359584)
How are BtDig crawling the DHT? I thought this was made impossible or at least more difficult a while ago.
Re:Ask HN: What is your favorite method of sending large files? by chid on Wednesday September 02, 2020 @09:20PM (#24360005)
I don't see how it was made impossible. It's definitely possible, you can do it yourself there's many examples on github (even btdig has an old version on it)
Re:Ask HN: What is your favorite method of sending large files? by 3np on Wednesday September 02, 2020 @08:28PM (#24359718)
Its trivial to disable PEX and DHT in any reasonable client