Distributing NixOS with IPFS

Re:Distributing NixOS with IPFS by whyrusleeping on Thursday January 19, 2017 @09:20PM (#13441305)

Hey! IPFS Dev here.

The cid stuff has been implemented and initial support for it is being landed in our 0.4.5 release (which will be soon, hopefully release candidate within a week).

With that and IPLD, you can craft arbitrary objects in JSON or CBOR (theres a 1 to 1 mapping, objects are stored as cbor) and work with them in ipfs. For example, i could make an object that looks like:

  {
    "Contents": {"/":"QmHashOfPkgContents"},
    "Compression": "bzip2",
    "NarSize": 12345,
    "References": {
      "foo": {"/": "QmHashOfFoo"},
      "bar": {"/": "QmHashOfBar"}
    },
    "Signature": "signature info, or a link to the signature",
  }

(please excuse my attempt at recreating a nar file in rough json).

This object could then be put into ipfs with:

  cat thing.json | ipfs dag put

And you would get an ipld object that you can move around in ipfs, and do fun things like:

  ipfs get <thatobjhash>/Contents

to download the package contents, or:

  ipfs get <thatobjhash>/References/foo

to get the referenced package (or open that hash/path in an ipfs gateway to browse the package graph for free in your browser :) )

[ Parent | on HN ]

Re:Distributing NixOS with IPFS by Ericson2314 on Thursday January 19, 2017 @10:42PM (#13441596)

IPLD does allow storing tons of data, but custom schemas allow restricting the data referenced in arbitrary ways.

IPLD, last I checked, supports relative paths (which can make certain cycles), and not every node child gets its own hash. This is too much flexibility for my purposes (Nix or otherwise).

Also, when interfacing with legacy systems like git repos, one needs to dereference a legacy hash without knowing what it points to, which is easiest done with custom schemas.

Now, granted, customs schemas aren't a super fine-grained solution as every node in the network that cares about the data needs to implement the schema, but they are useful tool for these reasons (and that downside doesn't apply to private networks).

[ Parent | on HN ]

Re:Distributing NixOS with IPFS by lgierth on Thursday January 19, 2017 @11:14PM (#13441688)

Also see the multicodec table for codes of ethereum/bitcoin/zcash/stellar

https://github.com/multiformats/multicodec/blob/2725f3c5cd7b...

[ Parent | on HN ]

Re:Distributing NixOS with IPFS by Ericson2314 on Friday January 20, 2017 @11:01AM (#13444695)

Ok, so it's good we can finally refer to other node types. But I worry about putting all that in a single namespace. The IPLD node types constitute different hashing strategies as I describe above, but stuff like media codecs are orthogonal to hashing strategies---media of various sorts given a hashing strategy will be treated as black-box binary data for the foreseeable future.

The big takeaway here is a really like the idea of IPFS, and want to be a full fan, but everywhere I look I see dubious interfaces. I see what already looks like legacy cruft, and they haven't even hit 1.0!

[ Parent | on HN ]

Re:Distributing NixOS with IPFS by Ericson2314 on Friday January 20, 2017 @11:12AM (#13444779)

Is there a reason git is not on this table yet?

[ Parent | on HN ]

Re:Distributing NixOS with IPFS by lgierth on Thursday January 19, 2017 @11:13PM (#13441683)

> Also, when interfacing with legacy systems like git repos, one needs to dereference a legacy hash without knowing what it points to, which is easiest done with custom schemas.

The CID (address format) in IPLD doesn't represent types of systems, but it represents types of data structures. E.g. in the case of git, it's not "git,$thehash", but instead "git-tree,$thehash" or "git-commit,$thehash".

That way you know which code you'll need to run once you have the object's payload, or you could have datastores that simply pull blocks out of a git repo.

Is this getting closer to what you mean?

[ Parent | on HN ]

Re:Distributing NixOS with IPFS by Ericson2314 on Friday January 20, 2017 @11:02AM (#13444702)

Yeah, my OP was saying what's happened to CID. I guess it's been implemented without finishing off the spec :/.

While I'm not opposed to treating git that way, do note that git hashes are specifically constructed by prefixing the serialization of blobs, trees, and commits separately so that collisions are not likely.

[ Parent | on HN ]