News for hackers. Stuff that matters.
Re:Deprecating XML by haberman on Thursday November 18, 2010 @04:32AM (#1917301)
> The protobuf encoding carries a ton of redundant crap around (if not as much as text formats).

[citation needed]

One of the features of protocol buffers is that they're backwards and forwards compatible. You can add and remove fields, change "required" to "optional" and back again, and still make sense of what comes to you on the wire. I don't think there's much of anything that can be eliminated from the protobuf binary format.

Eliminating tags for a field (even if you consider it required) wouldn't be backward compatible with a previous version of the protocol that considered it optional.

A message from a previous version of the schema is not "corrupted." Being able to make wire-compatible changes to the protocol is an extremely important feature.

Re:Deprecating XML by prodigal_erik on Friday November 19, 2010 @01:49AM (#1920675)
You can't add or remove required fields, because outdated recipients will be dangerously wrong about whether they understand the intent of a message in the revised format. At some point meeting new requirements and keeping interoperability calls for deprecating and replacing the old format, because treating everything as optional (which some Googlers apparently do, avoiding "required" completely) is almost as bad as having 2^n mostly-untested formats.
Re:Deprecating XML by haberman on Friday November 19, 2010 @03:31AM (#1920836)
This is the kind of over-engineering analysis that leads to overly complicated systems like XML schema.

Just because a field says "optional" doesn't mean it's logically optional. You don't have to make your schema formalism complex enough that it can describe every last rule of what it takes for a message to be valid. In fact you definitely don't want to do that, because it's a horrible amount of complexity in the schema for little gain.

Yes, it's true that some Googlers use "optional" instead of "required" everywhere in their .proto files. That doesn't mean that you can omit any field and expect your message to be processed by your peer without error. It just means that you won't get an error at the schema validation level. But the application could still throw an error. More complex rules about what fields must be specified or what values they must have can be described in comments, and enforced with custom validation if necessary.

Also, since protobufs support default values, you can define what value will be returned for scalar fields if no value is explicitly sent. This can often be used to define useful default behavior for the case where a field is omitted.

Re:Deprecating XML by prodigal_erik on Saturday November 20, 2010 @05:43PM (#1925583)
It comes down to different design choices. Any constraints that are missing from the schema have to be recreated separately in each implementation, and I see them as unlikely to do that consistently enough to interoperate well. So I prefer not to pay at runtime for expressing many variations of the protocol, when I don't expect them to work anyway. Like HTML vs. XHTML—I shudder to think how much work was wasted trying to handle the worst tag soup imaginable, simply for lack of a well-formedness (or DTD validity) requirement.
Re:Deprecating XML by haberman on Tuesday November 23, 2010 @05:33PM (#1935660)
> Any constraints that are missing from the schema have to be recreated separately in each implementation

Not true at all. The server can implement them -- once -- and any client who makes an invalid request to the server will get an error message. These constraints can be expressed in comments in the interface (.proto) file.

> Like HTML vs. XHTML—I shudder to think how much work was wasted trying to handle the worst tag soup imaginable, simply for lack of a well-formedness (or DTD validity) requirement.

HTML and XHTML is a completely different ball of wax. Insisting on even well-formedness is simply unreasonable in practice, because it is so difficult to ensure, and it is the user who pays the price when the software isn't perfect.

If you're still convinced that the world would have been better if strict XHTML had won, you should read: http://diveintomark.org/archives/2004/01/14/thought_experime...