News for hackers. Stuff that matters.
Re:Hashpipe – Pipe iff the hash matches by gojomo on Friday April 03, 2015 @07:07PM (#9319118)
When the just-over-the-next-hilltop promised-land nirvana of content-centric networking arrives, the hash will be enough to locate & download the content – so you shouldn't even need an URL:

  $ hashcurl QmUJPTFZnR2CPGAzmfdYPghgrFtYFB6pf1BqMvqfiPDam8 | sh
Maybe it's even a special filesystem path, that contains (but does not list) everything-that's-nameable-and-findable:

  $ sh /everything/QmUJPTFZnR2CPGAzmfdYPghgrFtYFB6pf1BqMvqfiPDam8
(BTW, personally not a fan of the opaque 'multihash' format, which obscures the algorithm-in-use to save a few characters.)
Re:Hashpipe – Pipe iff the hash matches by _prometheus on Friday April 03, 2015 @07:33PM (#9319213)
this already works. install ipfs: http://ipfs.io/docs/install then:

    ipfs init
    ipfs daemon &
    sleep 20  # sorry this will go away
    ipfs mount
    sh /ipfs/QmTpnQL97XEHmyt54mgEwf5BN8gJWvw4sGgwQzjqtBwLX6
you can see it on the web at: http://gateway.ipfs.io/ipfs/QmTpnQL97XEHmyt54mgEwf5BN8gJWvw4...
Re:Hashpipe – Pipe iff the hash matches by KMag on Sunday April 05, 2015 @05:47AM (#9323605)
If any of you are designing a system like ipfs, please use Merkle tree roots as the identifiers (and make sure leaf nodes are hashed differently than inner nodes as in THEX, or better yet Joan Daemen's Sakura construction).

The main reason is that at some point you probably want to support downloading from multiple untrusted sources. Properly implemented Merkle trees (such as using the Sakura construction) are provably as strong as the underlying hash algorithm and allow lightweight cryptographic proof that a given block belongs to the file in question and belongs in the claimed place.

Gnutella uses the SHA-1 of the whole file the identifier and then just assumes the first peer to give it a Merkle tree root is giving it the correct root. Because there isn't guaranteed consistency between the identifier and the tree root, this makes it vulnerable to an attack where an attacker tries hard to be first, and give it a root for a corrupted version of the file and the peer wastes a lot of time and bandwidth re-downloading uncorrupted blocks that fail to verify.

Alternatively, you could make the identifier the concatenation of a full-file hash and a Merkle tree root. However, since the Sakura construction (and several other constructions) are provably as strong as the underlying hash, this is a waste of space. Concatenating the SHA-256 of the file with a SHA-256 Merkle tree root only gives you as much strength as the first 257 bits of a SHA-512 Merkle tree root. If you can spare the space for the longer identifiers, you're better off using a longer hash instead of two shorter hashes.

In other words, it's a waste of space to concatenate a whole-file hash with a Merkle tree root if you use a tree construction that's provably as strong as the underlying hash function.

Re:Hashpipe – Pipe iff the hash matches by gojomo on Friday April 03, 2015 @07:41PM (#9319234)
That's great! I hope that IPFS, or something of its ilk, can be the promised-land to which I alluded.

If I were to install it, how hard/breaking would it be to change the access-path to something more grandiosely descriptive like '/everything/'?

Re:Hashpipe – Pipe iff the hash matches by whyrusleeping on Friday April 03, 2015 @08:31PM (#9319379)
its a simple matter of changing the ipfs config file
Re:Hashpipe – Pipe iff the hash matches by _prometheus on Friday April 03, 2015 @09:50PM (#9319574)
<3
Re:Hashpipe – Pipe iff the hash matches by _prometheus on Friday April 03, 2015 @09:50PM (#9319576)
also, we want to push towards `/` being everything, shoving all your local crap into `/local` or `/localhost`
Re:Hashpipe – Pipe iff the hash matches by serendipitous on Friday April 03, 2015 @10:11PM (#9319626)
This sounds like DHT, which has existed for a while now. https://en.wikipedia.org/wiki/Distributed_hash_table
Re:Hashpipe – Pipe iff the hash matches by stouset on Friday April 03, 2015 @07:26PM (#9319178)
It's not save a few characters. It's to allow for flexibility of encoding. "sha1" all but requires ASCII. Sometimes you're limited to hex, base64, etc.
Re:Hashpipe – Pipe iff the hash matches by gojomo on Friday April 03, 2015 @09:07PM (#9319466)
Not sure I understand; almost any context where "QmTpn…" can appear could also handle a "sha1:" human-readable prefix. Including, notably, this shell context.

If another context is sufficiently different, it would be fine to use a controlled fixed-length binary-vocabulary there. (But, the ASCII bytes for "sha1:" would still be a pretty robust and largely self-documenting choice.)

Re:Hashpipe – Pipe iff the hash matches by _prometheus on Friday April 03, 2015 @11:59PM (#9319843)
oh the "Qm..." string is the base58 representation. you can have a binary packed version as well. check out more at https://github.com/jbenet/multihash/

the important thing here is standardizing the encoding, i.e. there should be a valid repr in {binary, hex, b32, b58, b64, ... bemoji ...}

so that, say, if your input field _only takes hex_ you have a way to enter the hash. this problem surfaces when tools expect reprs to be in a particular encoding (all too common), and you dont have power or access to make the tool better (sadly all too common too).

Re:Hashpipe – Pipe iff the hash matches by gojomo on Saturday April 04, 2015 @01:09AM (#9319977)
'sha1:' (0x736861313a) and other self-describing labels can be represented in hex (etc) just fine.
Re:Hashpipe – Pipe iff the hash matches by stouset on Saturday April 04, 2015 @02:37AM (#9320174)
So... it's readable in one out of dozens of possible encodings? And realistically, in that encoding (ASCII), the part following the readable prefix is unreadable garbage. In exchange for this convenience, you add a four bytes of unreadable prefix instead of one.

I'm not sure I see the point.

I deal with outputs from crypto functions on a daily basis. Never once have I intentionally rendered it as 8-byte ASCII. It's most often in hex. Mixing and matching encodings (e.g., prefixing a hex string with an ASCII string) for a single blob of data is silly and just causes headaches whenever you need to change encodings, either by having some data double-encoded or by having to specially handle the prefix separately.

Re:Hashpipe – Pipe iff the hash matches by gojomo on Saturday April 04, 2015 @03:46AM (#9320287)
My hunch is that the overwhelmingly dominant and important use case is where these identifiers appear in URLs, including URL fragments (and potentially in brand-new protocols). A few other important cases are also where they're visible to people, as in the example (or hypothetical) command-lines that kicked off this thread. In those cases, explicitness-at-a-glance helps, as will having one canonical encoding (such as b32, b64, or b58).

Compared to that, adaptation to other constrained systems is a case-by-case issue. And if those systems are already capable of squeezing in these non-native slightly-longer hash-like-strings, then a few more bytes usually won't hurt, or if they do then whatever deep-in-the-bits coder (like yourself) who's shoehorning things in can handle compactification. The display/exchange format should be as casually readable as possible.

Such an ASCII-name-inspired prefix is readable in all encodings. In some, it's just a magic number (but one that any coder can make educated guesses about); but in the one encoding that's likely most important for mutual comprehension between humans (user-exchanged strings and URLs), it's super-duper-readable.

And yes, the prefix should in general have special handling: as the multihash project README notes in an important "warning", the prefix lacks the same distribution as the other bytes. Treating the whole thing as an opaque-but-still-reliable identifier invites indexing misoptimizations right off the bat, and then other later bugs, if any of the hash functions become deprecated, or a new hash is added with variant semantics.

Re:Hashpipe – Pipe iff the hash matches by andrewflnr on Friday April 03, 2015 @07:22PM (#9319165)
I'm not sure if it's really great or a really terrible, but I love the "/everything" idea.
Re:Hashpipe – Pipe iff the hash matches by whyrusleeping on Friday April 03, 2015 @08:31PM (#9319382)
you should check out ipfs.io then :)

Disclaimer: I'm one of the devs