News for hackers. Stuff that matters.
Re:Understanding Serde by bestouff on Monday November 18, 2019 @04:23AM (#21562935)
Serde is one of Rust's killer feature which makes you appreciate the philosophy behind the language. With just a few annotations you have a robust and complete parser for whatever format (JSON, TOML, etc.) automatically built for serializing and deserializing your data structures. And it's fast, and optionnally zero-copy. I've never seen this in other languages.
Re:Understanding Serde by rkangel on Monday November 18, 2019 @05:41AM (#21563254)
It is best not to confuse deserialisation and parsing. Parsing is a separate task with separate issues.

For JSON deserialisation, Serde is a very convenient API to use an existing JSON parser that someone else has (hand) written.

If you needed a new syntax, you'd have to write a new parser. I often implement communications protocols as part of my work. In most cases, Serde doesn't help me much there - I have to parse the protocol myself (e.g. using Nom).

Re:Understanding Serde by legulere on Monday November 18, 2019 @08:21AM (#21564051)
What is the difference between parsing and deserializing? With both you have a string representation of a data structure that you read and bring into the native form. Do you call it parsing if the grammar is more complicated?
Re:Understanding Serde by p0llard on Monday November 18, 2019 @08:42AM (#21564204)
Deserialisation (in this context) is the act of decoding data from some "string representation" to a representation suitable for use, typically a native format.

Typically, although this depends on the specific encoding, a parser is required as one of the first stages of a deserialiser; normally a parser refers to something which applies the rules of something as least as complex as a CFG, as opposed to a lexer/scanner which typically just applies regular expressions; a combination of the two allows the serialised format to be interpreted.

You can show quite easily that JSON (and indeed any CFL) cannot be "parsed" using a lexer alone.

Re:Understanding Serde by empath75 on Monday November 18, 2019 @11:14AM (#21565422)
Serde includes a bunch of parsers by default but if you invented a new text file format, you’d have to write a new parser for it before you could do anything with it, and serde doesn’t help with that.
Re:Understanding Serde by specialist on Monday November 18, 2019 @09:28AM (#21564524)
Good question, thanks.

I had the vague notion that parsing was for human readable content and deserialization was for computer-to-computer content.

Now I tentatively make the distinction that compiling transforms streams into code (stream -> lex -> parse -> magic -> code) whereas deserialization is for data. Making parsing just one of the transformation steps for both compilation and deserialization.

Re:Understanding Serde by earthboundkid on Monday November 18, 2019 @08:37AM (#21564175)
Serde only helps with moving the values from the parser into a struct. It doesn't help with figuring out the data format.
Re:Understanding Serde by ncmncm on Monday November 18, 2019 @02:33PM (#21567377)
The really essential difference is that when you deserialize, you trust the input, and so don't need to check syntax or validity. I.e., you assume it is output of your own serializer, and you can therefore be faster than if you didn't trust it.

I have no idea if serde follows this design, or if it has just hijacked the terms.

Re:Understanding Serde by skocznymroczny on Monday November 18, 2019 @06:10AM (#21563367)
D also has serialization libraries based on introspection capabilities. Orange is an example of such library https://code.dlang.org/packages/orange . Here's a blog series (in progress) showing how to write a serializer/deserializer from scratch in D: https://bradley.chatha.dev/Home/Blog?post=JsonSerialiser1
Re:Understanding Serde by JoshMcguigan on Monday November 18, 2019 @07:36AM (#21563814)
Thanks for sharing. I'm not familiar with D, but it looks like compile time type introspection is something which is built into the language, which removes the need for something like the serde derive macro. Is that correct?
Re:Understanding Serde by skocznymroczny on Monday November 18, 2019 @09:33AM (#21564557)
Yes. D metaprogramming is mostly based around template and it gives you compile-time traits which you can use to query structs/classes for their fields, determining if something is an array or a pointer etc.

This way, you can write a serialize(foo) function which will work for any type, without needing any derives or marking the type as Serializable like in Java.

The closest equivalent to derive in D would be a mixin. You could declare a mixin which autogenerates the toJSON() fromJSON() methods based on the fields of a struct.

Re:Understanding Serde by chris_overseas on Monday November 18, 2019 @06:56AM (#21563581)
Kotlin serialization[0] is similar to this, implemented via a compiler plugin that allows optimal serializers/deserializers to be generated at compile time.

[0] https://github.com/Kotlin/kotlinx.serialization

Re:Understanding Serde by jcl on Monday November 18, 2019 @11:30AM (#21565597)
I've read, though, that one downside of Serde is that it tends to generate large binaries -- the cost of performance, I guess.

https://raphlinus.github.io/rust/2019/08/21/rust-bloat.html#...

Re:Understanding Serde by jimenezrick on Monday November 18, 2019 @05:30AM (#21563204)
Haskell? Data generics isn't really nothing new. We had this technology for years.
Re:Understanding Serde by nicoburns on Monday November 18, 2019 @10:56AM (#21565280)
True. Not widely available though. The great thing about Rust is that has a whole of the advantages of Haskell, and C++, and to a lesser extent even languages like Python.
Re:Understanding Serde by earthboundkid on Monday November 18, 2019 @08:40AM (#21564189)
Haskell has been around for more than a decade and had huge uptake by language nerds, but the only significant OSS written in it is Pandoc. In contrast, Rust already has Servo and a number of CLI tools, ripgrep being the most prominent.
Re:Understanding Serde by leshow on Monday November 18, 2019 @10:20AM (#21564930)
I'm not sure what you consider 'significant', but to my mind, there are many great haskell open source projects: xmonad, shellcheck, stack, haxl, to name a few.

Not to mention how many languages are written in it: Elm, Purescript, Idris, GHC itself.

Re:Understanding Serde by earthboundkid on Monday November 18, 2019 @12:59PM (#21566471)
I almost listed xmonad. I haven’t heard of the other ones. Doesn’t change my point.
Re:Understanding Serde by jimenezrick on Monday November 18, 2019 @11:58AM (#21565886)
Leaving popularity contests aside, i worked in a company using Haskell and i did used a lot these techniques, if that counts \o/

i guess today it only matter how many stars you repo has in github...

Re:Understanding Serde by bjz_ on Monday November 18, 2019 @06:16AM (#21563394)
At the same level of performance though?
Re:Understanding Serde by jimenezrick on Monday November 18, 2019 @07:25AM (#21563748)
If you are asking if it's fast? yes it's fast serialization. Not sure which numbers you have at hand, but for me fast means it isn't the bottleneck of my application.
Re:Understanding Serde by ghostwriter on Monday November 18, 2019 @07:29AM (#21563769)
it doesn't have to be to make the point, as Serde isn't on the same level of verbosity as Aeson.
Re:Understanding Serde by pjmlp on Monday November 18, 2019 @04:56AM (#21563058)
Common Lisp?
Re:Understanding Serde by fnord123 on Monday November 18, 2019 @05:08AM (#21563106)
Have you got a link to the Common Lisp library that lets you annotate data structures so you can use the annotations to parse or dump data by 3rd party libraries for various formats?
Re:Understanding Serde by pjmlp on Monday November 18, 2019 @05:22AM (#21563173)
No, but given that Common Lisp macro capabilities trump what Rust is capable of, that would be easily achievable.

Or now that I think of it, Template Haskell.