Re:JSON+UDP+DHT=Freedom
by keithwinstein on Thursday August 23, 2012 @11:15AM
(#4422806)
This is very interesting. We looked at storing stuff in BitTorrent's Kademlia DHT to do roaming NAT-traversal in Mosh, but their performance scared us off. Before building an application on this I would want to ask:
- Who runs telehash.org? Will they still be around in five years? Is there a well-known list of anchor nodes I can hardcode into my app, or how does the program find the DHT when it first starts?
- What is the median time to retrieve a key? What will it be in 24 months?
- What is the percentage of failed requests? Can we guess what that will be in 24 months?
The "mainline" BitTorrent Kademlia DHT started off great but now the statistics are apparently abysmal. In one academic study, the median time to retrieve a key was over a minute, and 20% of retrievals failed. (See http://www.cs.rice.edu/~scrosby/tr/BTMeasure-Main.pdf)
Why won't the same fate befall this Kademlia DHT?
(Edit: is this _A_ DHT, or is the idea that if you build your program to use Telehash, you are creating your own DHT that is only used by currently-running instances of the same program?)
Re:JSON+UDP+DHT=Freedom
by nmcfarl on Thursday August 23, 2012 @12:44PM
(#4423372)
> (Edit: is this _A_ DHT, or is the idea that if you build your program to use Telehash, you are creating your own DHT that is only used by currently-running instances of the same program?)
I had assumed the later, but the existence of telecast.org:42424 kind of points against it. It's possible that we've got both going on.
Re:JSON+UDP+DHT=Freedom
by towhans on Thursday August 23, 2012 @04:11AM
(#4421397)
I am a programmer yet still I don't get what this is. If an app wants to send JSON to another app why not use HTTP for that. DNS will take care of the routing. The only thing that seems novel to me is their use of UDP which is faster then HTTP.
Re:JSON+UDP+DHT=Freedom
by artumi-richard on Thursday August 23, 2012 @04:50AM
(#4421476)
Is it to bypass DNS? With all the moves to control the internet around, then an encrypted peer-to-peer decentralised system that can deal with any kind of data would allow for DNS-less email / websites etc etc...
Though the main point of DNS is it manages a namespace, and you wouldn't want clashes, so websites would probably have to have urls like thttp://{some 64char hash}/
Re:JSON+UDP+DHT=Freedom
by medecau on Thursday August 23, 2012 @06:04AM
(#4421620)
The use of UDP allows for "hole-punching".
User A wants to send and receive messages with user B.
Both are identified only by their IP and Port pair.
User B does not know of user As intentions.
So user A send switch C an UDP packet asking for "hole-punching".
Switch C has this service where users behind NAT routers connect and signup to be contacted when someone wants to "hole-punch" to them.
Switch C sends a packet to B with info about A.
B sends A a packet and Bs router is now expecting packets from A. Has soon has A sends its first packet to B its router is open for packets from B. Now both A and B can send packets to each other without C.
DNS is not yet entirely out of the loop.
The main switch for telehash is telehash.org:42424
Re:JSON+UDP+DHT=Freedom
by axx on Thursday August 23, 2012 @06:03AM
(#4421617)
From the Wiki:
[...]End: a SHA-1 hash key (40 hex characters, 160-bits) stored in the global DHT and distributed between switches. Switches will distribute and look up ends in the global DHT that are important to them. TeleHash defines the SHA-1 hash of the external IP:PORT of a switch as part of the protocol. Applications built on top of TeleHash can add their own ends to the DHT like hashes of files, hashes of e-mail addresses or hashes of other application-specific values (e.g. user@chat).[...]
So if i understand this right, every packet has a protocol-defined +end and the application can add it's own end on top (i.e. an Email adress?).
Re:JSON+UDP+DHT=Freedom
by arethuza on Thursday August 23, 2012 @06:08AM
(#4421631)
OK, I was being a bit thick there, to look up a email address you presumably simply hash it (presumably with some kind of canonicalization) and use that as the key for your lookup for a matching +end.
Re:JSON+UDP+DHT=Freedom
by axx on Thursday August 23, 2012 @06:36AM
(#4421703)
I'm not sure, but this is what i think i get out of this. You really don't need an email, this could be a hash or whatever. i think of more apps or services use some protocol like this, there should be a unified why to "lookup" for an account or something.
For example, if you have a decentralized social network that is based on the same software, say "facespace", every of those software nodes would lookup for the same identifier. i guess...
Re:JSON+UDP+DHT=Freedom
by paul-woolcock on Thursday August 23, 2012 @10:54AM
(#4422687)
Pendantic and off-topic, but it irks me when ZMQ gets lumped in with i.e. RabbitMQ and AMQ. ZMQ is more like an improved socket library and tools for building distributed message queues (among many other things), while RMQ and AMQ are actually message queue applications.
Re:JSON+UDP+DHT=Freedom
by sudhirj on Thursday August 23, 2012 @01:16AM
(#4421027)
I can just glimpse the promise this holds, but I don't really understand one would implement this. Does anyone have a blog post with example code?
If I've got a server that needs to receive messages, do I register it as an 'end'? And do I then send messages to this 'end' from other clients? How does it prevent two parties from registering the same 'end' hash?
Re:JSON+UDP+DHT=Freedom
by b3tta on Thursday August 23, 2012 @06:10AM
(#4421640)
To the "Why UDP?" part:
For things like P2P UDP is a much better fit, because TCP sends an ACK (ACKnowledgement) to the sender for every received packet.
Since UDP is unreliable no such ACKs will be send and it's up to the programmer to detect packet loss. The latency is therefore with UDP lower (and hence throughput higher), because the provider can push packets without waiting for a response.
For instance in P2P the receiver only needs to send a NAK (Negative AcKnowledgment) if and only if a packet/part is missing and then it will be resend by the provider.
Re:JSON+UDP+DHT=Freedom
by icebraining on Thursday August 23, 2012 @07:12PM
(#4425372)
TCP sends an ACK (ACKnowledgement) to the sender for every received packet.
Not quite. TCP receivers only need to send ACKs when they have received enough data to fill the sliding window. That may be every packet, every other packet (the recommended by the RFC) or more.
Re:JSON+UDP+DHT=Freedom
by popee on Thursday August 23, 2012 @10:42AM
(#4422611)
You could have problems with NAT, but SIP protocol has STUN and ICE which are responsible for dealing with this. Check that out.
Btw i had idea SIP + JSON/HTML + DHT which would not suffer with "man in the middle attack" that HTTP has as designed decision. You could connect, for example, with HTTP/FTP/... protocols to your buddies directly. You just have to make right SDP packet when calling them ...
Re:JSON+UDP+DHT=Freedom
by hdradionet on Thursday August 23, 2012 @08:51AM
(#4422055)
UDP+DHT+NAT Traversal is very powerful. I implemented a new p2p radio platform called HD Radio.NET (http://hdradionet.zapto.org/) which is completely based on them. It shows to be very effective. In this news, Jason is a new idea, it may bring some benefits, for example, running something in browser.
Re:JSON+UDP+DHT=Freedom
by alberich on Thursday August 23, 2012 @06:05AM
(#4421622)
I was wondering how this approach deals with systems running behind NAT also. With IPv6 not happening anytime soon, there will be a large number of systems without individual public addresses available. Routing this UDP packets may require tweaking firewall rules, etc.
Re:JSON+UDP+DHT=Freedom
by arnoldb0620 on Thursday August 23, 2012 @08:07AM
(#4421903)
if server a is behind a nat, it first needs to send out a message to server b this will open a random port for typically 60 seconds automatically. Then another message will need to originate from server a within the 60 second time period in order to keep the port open.
- Who runs telehash.org? Will they still be around in five years? Is there a well-known list of anchor nodes I can hardcode into my app, or how does the program find the DHT when it first starts?
- What is the median time to retrieve a key? What will it be in 24 months?
- What is the percentage of failed requests? Can we guess what that will be in 24 months?
The "mainline" BitTorrent Kademlia DHT started off great but now the statistics are apparently abysmal. In one academic study, the median time to retrieve a key was over a minute, and 20% of retrievals failed. (See http://www.cs.rice.edu/~scrosby/tr/BTMeasure-Main.pdf)
Why won't the same fate befall this Kademlia DHT?
(Edit: is this _A_ DHT, or is the idea that if you build your program to use Telehash, you are creating your own DHT that is only used by currently-running instances of the same program?)
I had assumed the later, but the existence of telecast.org:42424 kind of points against it. It's possible that we've got both going on.
Though the main point of DNS is it manages a namespace, and you wouldn't want clashes, so websites would probably have to have urls like thttp://{some 64char hash}/
(Where the h stands for telehash...
User A wants to send and receive messages with user B. Both are identified only by their IP and Port pair. User B does not know of user As intentions. So user A send switch C an UDP packet asking for "hole-punching". Switch C has this service where users behind NAT routers connect and signup to be contacted when someone wants to "hole-punch" to them. Switch C sends a packet to B with info about A. B sends A a packet and Bs router is now expecting packets from A. Has soon has A sends its first packet to B its router is open for packets from B. Now both A and B can send packets to each other without C.
DNS is not yet entirely out of the loop. The main switch for telehash is telehash.org:42424
i.e. Map from something user friendly (e.g. username) to a SHA1 hash?
[...]End: a SHA-1 hash key (40 hex characters, 160-bits) stored in the global DHT and distributed between switches. Switches will distribute and look up ends in the global DHT that are important to them. TeleHash defines the SHA-1 hash of the external IP:PORT of a switch as part of the protocol. Applications built on top of TeleHash can add their own ends to the DHT like hashes of files, hashes of e-mail addresses or hashes of other application-specific values (e.g. user@chat).[...]
So if i understand this right, every packet has a protocol-defined +end and the application can add it's own end on top (i.e. an Email adress?).
https://github.com/quartzjer/TeleHash/wiki/My-Understanding-...
For example, if you have a decentralized social network that is based on the same software, say "facespace", every of those software nodes would lookup for the same identifier. i guess...
{ "facespace":"arethuza" }
With full decentralization it could really become a true 'protocol'.
...until they need money.
If I've got a server that needs to receive messages, do I register it as an 'end'? And do I then send messages to this 'end' from other clients? How does it prevent two parties from registering the same 'end' hash?
http://telehash.org/v2.html
For things like P2P UDP is a much better fit, because TCP sends an ACK (ACKnowledgement) to the sender for every received packet.
Since UDP is unreliable no such ACKs will be send and it's up to the programmer to detect packet loss. The latency is therefore with UDP lower (and hence throughput higher), because the provider can push packets without waiting for a response.
For instance in P2P the receiver only needs to send a NAK (Negative AcKnowledgment) if and only if a packet/part is missing and then it will be resend by the provider.
Not quite. TCP receivers only need to send ACKs when they have received enough data to fill the sliding window. That may be every packet, every other packet (the recommended by the RFC) or more.
Btw i had idea SIP + JSON/HTML + DHT which would not suffer with "man in the middle attack" that HTTP has as designed decision. You could connect, for example, with HTTP/FTP/... protocols to your buddies directly. You just have to make right SDP packet when calling them ...
In you FAQ it says it is cross platform, but there is only Windows software on the download page.
What about switches that want to run from behind a NAT? Could it be that UDP was chosen with this in mind? (http://en.wikipedia.org/wiki/UDP_hole_punching)
http://en.wikipedia.org/wiki/NAT_traversal