News for hackers. Stuff that matters.
Re:Scuttlebutt, a Decentralized Alternative to Facebook by staltz on Thursday April 19, 2018 @04:29PM (#16879833)
Re:Scuttlebutt, a Decentralized Alternative to Facebook by Vendan on Thursday April 19, 2018 @04:41PM (#16879954)
(I'm the author of the repo in question)

Note that this repo has been sidelined, as I have fundamental issues with the protocol SSB is built on. Unless there's changes to how the messages are signed and verified, I'm not planning on putting any serious effort into SSB.

Re:Scuttlebutt, a Decentralized Alternative to Facebook by lgierth on Thursday April 19, 2018 @04:47PM (#16880019)
I'm curious to hear about the issues. SSB's network transport is supposed to be pretty okay: https://github.com/auditdrivencrypto/secret-handshake
Re:Scuttlebutt, a Decentralized Alternative to Facebook by Vendan on Thursday April 19, 2018 @04:52PM (#16880060)
example message:

    {
      "previous": "%26AC+gU0t74jRGVeDY01...MnutGGHM=.sha256",
      "author": "@hxGxqPrplLjRG2vtjQL87...0nNwE=.ed25519",
      "sequence": 216,
      "timestamp": 1442590513298,
      "hash": "sha256",
      "content": {
        "type": "vote",
        "vote": {
          "link": "%WbQ4dq0m/zu5jxll9zUb...KjZ80JvI=.sha256",
          "value": 1
        }
      }
    }
Signed example message:

    {
      "previous": "%26AC+gU0t74jRGVeDY01...MnutGGHM=.sha256",
      "author": "@hxGxqPrplLjRG2vtjQL87...0nNwE=.ed25519",
      "sequence": 216,
      "timestamp": 1442590513298,
      "hash": "sha256",
      "content": {
        "type": "vote",
        "vote": {
          "link": "%WbQ4dq0m/zu5jxll9zUb...KjZ80JvI=.sha256",
          "value": 1
        }
      }
      "signature": "Sjq1C3yiKdmi1TWvNqxI...gmAQ==.sig.ed25519"
    }
The signature covers the whole message, and is then added into the message it signed. The way this works means you have to encode json exactly the same way as the "main" node.js implementation. Emojis, unicode, html literals, EVERYTHING. Or else it will fail to verify. I've gotten most of it working, but there's still edge cases where I gave up trying to get them to work correctly.
Re:Scuttlebutt, a Decentralized Alternative to Facebook by zeveb on Thursday April 19, 2018 @05:25PM (#16880304)
I know that I sound like a broken record, but this is exactly the issue which canonical S-expressions were designed for, and which SPKI wrestled with & solved twenty years ago.

The SPKI version of a message would look something like (I've removed the hash property, because I don't think it makes sense for an object to specify the hash to be used to refer to it, but one could add it back in if one wished):

    (message
     (previous (hash sha256 |XphMUkWQtomKjXQvFGfsGYpt69sgEY7Y4Vou9cEuJho=|))
     (author (public-key (ed25519 |FCX/tsDLpubCPKKfIrw4gc+SQkHcaD17s7GI6i/ziWY=|)))
     (sequence 216)
     (timestamp "2018-04-19T17:53:26Z")
     (content (type vote)
              (link (hash sha256 |DlBH/hCmXfVzks2uY+WIll4aTzxrfBA8m/3GIdX3Vew=|))
              (value 1)))
The transport version would be this (you can Base64-decode it to see the canonical version):

    {KDc6bWVzc2FnZSg4OnByZXZpb3VzKDQ6aGFzaDY6c2hhMjU2MzI6XphMUkWQtomKjXQvFGfsGYpt
    69sgEY7Y4Vou9cEuJhopKSg2OmF1dGhvcigxMDpwdWJsaWMta2V5KDc6ZWQyNTUxOTMyOhQl/7bA
    y6bmwjyinyK8OIHPkkJB3Gg9e7OxiOov84lmKSkpKDg6c2VxdWVuY2UzOjIxNikoOTp0aW1lc3Rh
    bXAyMDoyMDE4LTA0LTE5VDE3OjUzOjI2WikoNzpjb250ZW50KDQ6dHlwZTQ6dm90ZSkoNDpsaW5r
    KDQ6aGFzaDY6c2hhMjU2MzI6DlBH/hCmXfVzks2uY+WIll4aTzxrfBA8m/3GIdX3VewpKSg1OnZh
    bHVlMToxKSkp}
And a signed message might look something like:

    (sequence
     (message
      (previous (hash sha256 |XphMUkWQtomKjXQvFGfsGYpt69sgEY7Y4Vou9cEuJho=|))
      (author (public-key (ed25519 |FCX/tsDLpubCPKKfIrw4gc+SQkHcaD17s7GI6i/ziWY=|)))
      (sequence 216)
      (timestamp "2018-04-19T17:53:26Z")
      (content (type vote)
               (link (hash sha256 |DlBH/hCmXfVzks2uY+WIll4aTzxrfBA8m/3GIdX3Vew=|))
               (value 1)))
     (signature (hash sha256 |XphMUkWQtomKjXQvFGfsGYpt69sgEY7Y4Vou9cEuJho=|)
                (hash sha256 |5hHMWc1PqfrwFVfALci5JXCWqW7VC4I4iS4+Utvr44w=|)
                |z7W1ERg9UYZjNfE72ZwEuJF79khG+eOHWFp6iF+KLuSrw8Lqa6IousK4cCn9T5qFa8E14GVek4cAMmMbjqDnAg==|))
Canonical S-expressions already buy you bit-for-bit identity when hashing, and SPKI (as an example) wraps signatures rather than injecting them — the only sane choice.

Why do I bring this up? Obviously it's possible to make JSON be a cryptographically-sound format (either by foregoing objects for arrays, or by rules around object-field ordering, along with other rules about encoding), but using it instead of an already-sound format indicates an unfamiliarity with prior work in the field.

Re:Scuttlebutt, a Decentralized Alternative to Facebook by zeveb on Friday April 20, 2018 @10:42AM (#16885140)
One slight correction: I realised later on that in the current draft spec, unquoted tokens must not start with a digit. Thus the example would be:

    (message
     (previous (hash sha256 |XphMUkWQtomKjXQvFGfsGYpt69sgEY7Y4Vou9cEuJho=|))
     (author (public-key (ed25519 |FCX/tsDLpubCPKKfIrw4gc+SQkHcaD17s7GI6i/ziWY=|)))
     (sequence "216")
     (timestamp "2018-04-19T17:53:26Z")
     (content (type vote)
              (link (hash sha256 |DlBH/hCmXfVzks2uY+WIll4aTzxrfBA8m/3GIdX3Vew=|))
              (value "1")))
Re:Scuttlebutt, a Decentralized Alternative to Facebook by dragonwriter on Friday April 20, 2018 @11:31AM (#16885590)
Perhaps more to the point, the need for canonicalization in this use case is well understood from not only canonical S-expressions, but similar things done in XML and other cryptographically signed structured data formats. While not using one of the existing formats is not necessarily a bad thing, overlooking the well-established need for canonicalization is quite bad.
Re:Scuttlebutt, a Decentralized Alternative to Facebook by Vendan on Thursday April 19, 2018 @05:31PM (#16880336)
Agree completely. I don't know if I'd choose s-expr, but at the bare minimum, wrapping with signatures is the sane thing to do.
Re:Scuttlebutt, a Decentralized Alternative to Facebook by vvanders on Thursday April 19, 2018 @11:03PM (#16881881)
Yup, I've got an implementation of http over packet radio I've been working through and came to the same conclusion. Header contains the signature, base64 the message for verification.

Even though it's super low bitrate(1200bps/9600bps) since you have a binary foundation w/ base64 you can send raw binary to get low over-head but then easily verify+decode since base64 is well supported across a wide range of languages.

Re:Scuttlebutt, a Decentralized Alternative to Facebook by walterbell on Thursday April 19, 2018 @05:59PM (#16880544)
Which large scale open-source or commercial software is using s-expr today?
Re:Scuttlebutt, a Decentralized Alternative to Facebook by bwbw223 on Thursday April 19, 2018 @07:49PM (#16881148)
Re:Scuttlebutt, a Decentralized Alternative to Facebook by _emacsomancer_ on Thursday April 19, 2018 @07:04PM (#16880935)
Hacker News. Emacs. Anything Lispy.
Re:Scuttlebutt, a Decentralized Alternative to Facebook by duncan_bayne on Thursday April 19, 2018 @06:28PM (#16880731)
Why do you ask?
Re:Scuttlebutt, a Decentralized Alternative to Facebook by hathawsh on Thursday April 19, 2018 @05:12PM (#16880226)
Wow, that hurts. What solutions have you suggested to the project?
Re:Scuttlebutt, a Decentralized Alternative to Facebook by Vendan on Thursday April 19, 2018 @05:23PM (#16880296)
I had expressed interest in changing it, but it would require changing the core data structure the network is built on, and basically be a breaking change that would almost inevitably result in having to start the entire social web over from scratch. Didn't see huge interest from the devs in doing so.
Re:Scuttlebutt, a Decentralized Alternative to Facebook by hathawsh on Thursday April 19, 2018 @05:51PM (#16880498)
It sounds like the best path forward might be to document the NodeJS serialization format in perfect detail and create a test suite to verify all known corner cases. Until that's done, alternative implementations will be unreliable.
Re:Scuttlebutt, a Decentralized Alternative to Facebook by 3eto on Friday April 20, 2018 @06:13AM (#16883571)
What are the potential downsides of not changing it?
Re:Scuttlebutt, a Decentralized Alternative to Facebook by Vendan on Friday April 20, 2018 @11:26AM (#16885538)
Makes it difficult to implement the protocol in anything but node.js (and potentially version specific, if node.js ever decides to tweak json serialization stuff)
Re:Scuttlebutt, a Decentralized Alternative to Facebook by lgierth on Friday April 20, 2018 @12:03PM (#16885907)
Objects in JSON are per the spec unordered, while the signature scheme here most likely relies on the order of the object's key-value pairs.

Other data formats that are ordered, and serialize somewhat more deterministically, are CBOR, Protobufs, or s-exp.

Re:Scuttlebutt, a Decentralized Alternative to Facebook by tasmg on Friday April 20, 2018 @02:39AM (#16882626)
This is the same reason I lost interest. I may have stuck with it if the Rust crates weren't AGPL/GPL. (And thus can't be used in Apple's app-store which is an interesting target to me).
Re:Scuttlebutt, a Decentralized Alternative to Facebook by wjh_ on Thursday April 19, 2018 @04:46PM (#16880000)
Would you be able to elaborate on these issues?
Re:Scuttlebutt, a Decentralized Alternative to Facebook by Vendan on Thursday April 19, 2018 @04:53PM (#16880065)
did so in sibling comment