Re:Updating Torrents via DHT Mutable Items
by rakoo on Thursday August 11, 2016 @06:20AM
(#12267132)
There was btdigg at some point (https://btdigg.org/) but for some reason it's down today. The approach is not particularly complicated, there even were papers about that:
* In Vuze: https://www.usenix.org/legacy/event/woot10/tech/full_papers/..., slides at https://www.defcon.org/images/defcon-18/dc-18-presentations/...
* In Mainline: https://arxiv.org/ftp/arxiv/papers/1009/1009.3681.pdf
Basically all you have to do is sit there, people will come to you asking for content, you'll ask as a standard DHT node but note that the infohash exists, and get more information (name, number of peers, ...) about it in the background. All you need is a few IPs.
Re:Updating Torrents via DHT Mutable Items
by aninhumer on Thursday August 11, 2016 @07:45AM
(#12267402)
Ah yes, BTDigg was the name.
I'd imagine to do more thorough scraping you'd want to reconnect to the network repeatedly with different peer hashes, otherwise you'd only see the infohashes "near" one point in the network.
Re:Updating Torrents via DHT Mutable Items
by rakoo on Thursday August 11, 2016 @05:49PM
(#12271840)
Yep, that's the idea: if you join the DHT network with multpile, far away ids, you get more requests coming in. Unfortunately there is BEP 42 (http://www.bittorrent.org/beps/bep_0042.html) which restricts the number of IDs you can have on the same IP (the restriction is there for a good reason, so it's a good think to follow it).
Something like that would make it easy to create centralised search engines for casual users, and allow power users to run local databases.