News for hackers. Stuff that matters.
Re:The Protobuf Language Specification by CobrastanJorji on Monday September 12, 2022 @02:31PM (#32814726)
This is interesting. So one company invents and maintains a compiler, then a different company found that documentation to be insufficient (big surprise), so wrote out a lengthy standard that conformed to what the first company's compiler happened to do? Seems very useful, but also seems risky and hard to maintain. What happens when Google tightens a constraint or adds a new feature next week?
Re:The Protobuf Language Specification by Master_Odin on Monday September 12, 2022 @02:45PM (#32814931)
I think the hope is that this could be a situation like CommonMark and Markdown, where google's implementation will continue to exist, but that the community just totally moves over to this new specification / tooling, and that anytime someone says "protobuf", they don't even necessarily realize that Google has a thing, they just know this specification.
Re:The Protobuf Language Specification by smcl on Monday September 12, 2022 @05:19PM (#32817145)
Trouble is that I don’t think everyone’s moved on with Markdown, I still encounter different dialects that are subtly different. Even within one suite of tools from a single vendor … [angry stares in the direction of Atlassian]
Re:The Protobuf Language Specification by numbsafari on Monday September 12, 2022 @05:00PM (#32816898)
... except for whenever you've got to communicate or exist inside the Google ecosystem. Now you've got "protobuf as per Google" and "protobuf as per internet randos" with twice the dependency graph.

It's unfortunate, and not surprising, that Google hasn't made a protobuf spec. I realize there are a ton of protobuf fans out there but, personally, it just feels like a massive, awkwardly maintained mess that I'm forced to live with for (mostly) their benefit.

Re:The Protobuf Language Specification by jhumphries131 on Monday September 12, 2022 @03:11PM (#32815291)
We keep an eye on the protobuf repo, so if a change is made we can both incorporate it into our products (https://buf.build) and into this spec.

A great outcome for the ecosystem would be that Google chooses to engage with the community before making language changes. And the best possible outcome would be that the authority is eventually inverted: a specification doc becomes the definitive source of truth on the language and `protoc` is updated to conform to it, instead of vice versa.

Re:The Protobuf Language Specification by eklitzke on Monday September 12, 2022 @03:06PM (#32815224)
Generally new features in protobuf are just new features, so if a new feature is added then the worst case is the documentation will be out of date/incomplete for a short period of time.

For the most part the "constraint tightening" thing doesn't affect the language specification, at least in my experience. For example, there have been some changes in protobuf that affect things like serialization order. A change like that can break brittle tests that do things like checking that a function produces some exact serialized string/byte sequence, but they don't affect the semantics of the language.

Re:The Protobuf Language Specification by wrs on Monday September 12, 2022 @04:59PM (#32816884)
This is a similar situation to Ruby and Rubinius (an alternative implementation of Ruby). Because there was no Ruby specification other than the original MRI implementation, the Rubinius project (a new alternative implementation) created their own test suite to codify expected Ruby behavior. However, the MRI developers didn't use it, and the behavior diverged.

The original creator gave up on the idea [0] but it was immediately taken over by others and is still maintained [1]. In case of conflict, though, Matz (lead developer on MRI), not the specification, is the source of truth [2].

[0] https://github.com/rubinius/rubinius-website-archive/blob/87...

[1] https://github.com/ruby/spec

[2] http://ruby.github.io/rubyspec.github.io/bugs_found/

Re:The Protobuf Language Specification by mmastrac on Monday September 12, 2022 @02:36PM (#32814796)
(deleted)
Re:The Protobuf Language Specification by jhumphries131 on Monday September 12, 2022 @02:50PM (#32815000)
Matt, that is not our intention. But we are trying to build a business around making schema-driven APIs easy, and protobuf is at the core of our current products. So we are trying to improve the ecosystem around protobuf, and a critical aspect of that in our esteem is having a spec.

While `protoc` remains the source of truth, this spec captures the syntax and rules accepted and enforced by `protoc` in a far more detailed way than the official developer site.

Re:The Protobuf Language Specification by mmastrac on Monday September 12, 2022 @03:14PM (#32815325)
Ah, hey Josh. I didn't realize you were involved in this effort. FWIW, seeing your name associated with this definitely gives it a bit more authority than I had originally assumed.
Re:The Protobuf Language Specification by kentonv on Monday September 12, 2022 @04:19PM (#32816319)
If it helps, I (original author of proto2) have been advising Buf and like what they're doing. (Disclosure: I also invested a small amount.)

Buf is founded by engineers who spent a LOT of time working with Protobufs outside of Google. I was always the one saying "please don't write your own .proto parser" but I am convinced Buf actually knows what they are doing here and probably have all the details right.

Our industry has a whole lot of tooling and infrastructure built around JSON, and almost every piece of it could be way better if it were operating with well-defined types instead, in the same way that TypeScript tooling benefits vs. JavaScript. Google has had an all-protobuf ecosystem internally for a long time, but much of it will never be released publicly. So that leaves someone like Buf to really build it out. I'm pretty interested to see where they're able to take it.

Re:The Protobuf Language Specification by CobrastanJorji on Monday September 12, 2022 @02:47PM (#32814962)
Ahhh, is that the play? That makes a lot of sense.