News for hackers. Stuff that matters.
Re:Scuttlebot: Peer-to-peer database, identity provider, and... by sneak on Saturday April 18, 2020 @04:43PM (#22910941)
When I was faced with this (signing a structure), I serialized the json into base64, then put that base64 string as a value (along with the MAC) into a new json document. It of course increases deserialization overhead (json, verify, unbase64, inner json) but sidesteps this issue.

I thought about sorting keys and other things like that, and the dozen edge cases and potential malleability issues dissuaded me for the compatibility issues mentioned above.

How have others solved it?

Re:Scuttlebot: Peer-to-peer database, identity provider, and... by rakoo on Saturday April 18, 2020 @10:22PM (#22912932)
Use Bencoding, like bittorrent does: https://en.wikipedia.org/wiki/Bencode

As I put in another comment, a torrent id is a hash of a map, where one of the keys contains binary data. bencoding solved that decades ago already.

Re:Scuttlebot: Peer-to-peer database, identity provider, and... by aabbcc1241 on Saturday April 18, 2020 @11:07PM (#22913177)
It looks similar to stackish and BON (binary object notation) https://github.com/bon-org/bon-doc/blob/master/README.asciid...

BON is compatible with json+ and erlang data type, in specific, it allows any data type for the map key. Json only allows string as map key.

Re:Scuttlebot: Peer-to-peer database, identity provider, and... by sneak on Sunday April 19, 2020 @11:21AM (#22916483)
Why bencoding and not BSON or CBOR or any of the other serialization options?
Re:Scuttlebot: Peer-to-peer database, identity provider, and... by flir on Saturday April 18, 2020 @05:17PM (#22911161)
Binary protocols. Or out-of-band signing.
Re:Scuttlebot: Peer-to-peer database, identity provider, and... by elviejo on Saturday April 18, 2020 @06:46PM (#22911748)
Do you mind going deeper here?

Which protocols? Can you point to examples?

Because I was about to design a signing Json solution but based on the comments here it is a bad idea.

Re:Scuttlebot: Peer-to-peer database, identity provider, and... by zenhack on Saturday April 18, 2020 @09:03PM (#22912548)
The signing schemes I've seen used in binary protocols fall into two categories:

1. Canonicalize and sign: the format has a defined canonical form. Convert to that before doing cryptographic operations. If the format is well designed around it, this is doable, whereas JSON doesn't really have this and with many libraries it's hard to control the output to the degree that you'd need. 2. Serialize and sign: serialize the data as flat bytes, and then your signed message just has a "bytes" field that is the signed object. This is conceptually not far off from the base64 solution above, except that there's not extra overhead, since with a binary protocol you'll have a length prefix instead of having to escape stuff.

Re:Scuttlebot: Peer-to-peer database, identity provider, and... by bmm6o on Saturday April 18, 2020 @09:09PM (#22912585)
Being able to separate the object and signature saves tons of trouble https://latacora.micro.blog/2019/07/24/how-not-to.html
Re:Scuttlebot: Peer-to-peer database, identity provider, and... by moojd on Saturday April 18, 2020 @07:10PM (#22911889)
Protobuf, cap n proto, and messagepack are a few I've seen before
Re:Scuttlebot: Peer-to-peer database, identity provider, and... by boomlinde on Monday April 20, 2020 @05:52AM (#22922506)
It sounds like base64 was unnecessary there since a JSON string can contain serialized JSON.

Personally I'll just concatenate the values in a defined order and sign/hash that.

Re:Scuttlebot: Peer-to-peer database, identity provider, and... by sneak on Monday April 20, 2020 @05:59AM (#22922531)
jsonrpc “args” can vary in name, number, and type.
Re:Scuttlebot: Peer-to-peer database, identity provider, and... by boomlinde on Monday April 20, 2020 @07:09AM (#22922832)
Sure, but where you're doing

    {"msg": "eyJhIjogMTAsICJjIjogInRlc3RpbmciLCAiYiI6ICJoZWxsbyJ9", "h": "..."}
you can as well be doing

    {"msg": "{\"a\": 10, \"c\": \"testing\", \"b\": \"hello\"}", "h": "..."}
and skip base64 altogether.

If you mean this as a point against the second part of my post, it's of course only in some limited circumstances you can simply dump the values in a defined order and be done with it. To make it general, you have to have delimiters for at least arrays, objects and numbers, canonicalize number representations, and probably output the keys as well, at which point you've invented your own complete serialization protocol.

Re:Scuttlebot: Peer-to-peer database, identity provider, and... by VWWHFSfQ on Saturday April 18, 2020 @06:56PM (#22911812)
I just use JWTs whenever I have to pass signed messages.