Re:MessagePack: Fast and Compact Serialization
by ericmoritz on Monday August 15, 2011 @10:15AM
(#2886965)
This is schemaless. Protobufs require a .proto file that describes the data being serialized. A .proto file serves a similar purpose as the `struct` keyword in C.
MessagePack uses type prefixes to describe the data. For instance an unsigned 8-bit int starts with a `0xcc` followed by a byte.
Re:MessagePack: Fast and Compact Serialization
by frsyuki on Monday August 15, 2011 @11:47AM
(#2887353)
Compatibility with JSON is a characteristic of MessagePack.
If you're already using JSON, you can use MessagePack as a faster/smaller substitution of JSON.
Since Protocol Buffers brings their original semantics, you must change lots of codes.
Re:MessagePack: Fast and Compact Serialization
by rabidsnail on Monday August 15, 2011 @01:53PM
(#2887924)
This appears to be mmap-friendly. Since arrays have length values it's possible to jump around in a large messagepack-serialized structure without having to scan it. In JSON if you want to get to element 2 of an array you have to scan to the end of element 1. If there's a length value at the beginning of element 1 (as it looks like there is with msgpack) then you can seek straight past it.
Re:MessagePack: Fast and Compact Serialization
by roel_v on Monday August 15, 2011 @09:24AM
(#2886744)
Depends on how you define 'better', I guess - at the very least it has to be several magnitudes faster, with no overhead for compression, parsing, etc.
Re:MessagePack: Fast and Compact Serialization
by frsyuki on Monday August 15, 2011 @12:00PM
(#2887414)
You can use type-conversion APIs with MessagePack if you use C++, Java or C#. It converts deserialized objects (=dynamically-typed) into statically-typed objects using templates (C++) or reflection + dynamic code generation (Java and C#). You don't have to write boring type-checking codes.
Note that this is a characteristic of the implementation, not of the data format.
Some libraries for JSON like JSONIC (Java) provides these APIs.
Re:MessagePack: Fast and Compact Serialization
by frsyuki on Monday August 15, 2011 @11:19AM
(#2887235)
It works very well at least on ARM.
I've used MessagePack on iPhone 3GS for the format of a dictionary file (> 100MB). I utilized mmap and zero-copy deserialization feature of MessagePack.
And I received (and merged) some patches sent by those who use it on iPhone4/iPad2.
I've never tried to run it on MIPS. But it probably works if you use gcc.
Also, due to V8's poor C++ API, msgpack doesn't support deserialization of 64 bit integers.
JavaScript _does_not_ have a number type wide enough to hold 64-bit integer. V8 API has nothing to do with it.
MessagePack uses type prefixes to describe the data. For instance an unsigned 8-bit int starts with a `0xcc` followed by a byte.
Since Protocol Buffers brings their original semantics, you must change lots of codes.
I wrote about the pros/cons of schema here: http://news.ycombinator.com/item?id=2836242
Note that this is a characteristic of the implementation, not of the data format. Some libraries for JSON like JSONIC (Java) provides these APIs.
How well does it work on ARM/MIPS?
I've used MessagePack on iPhone 3GS for the format of a dictionary file (> 100MB). I utilized mmap and zero-copy deserialization feature of MessagePack. And I received (and merged) some patches sent by those who use it on iPhone4/iPad2.
I've never tried to run it on MIPS. But it probably works if you use gcc.
(I'm author of MessagePack)