News for hackers. Stuff that matters.
Re:gRPC-Go Engineering Practices by dmayle on Thursday January 25, 2018 @03:16PM (#16234201)
Json is a serialization format. gRPC is both a serialization format and a DDL (data definition language).

That means that you are storing your schema, which also happens to contain interoperability features.

...and the serialization format is more efficient.

Re:gRPC-Go Engineering Practices by Xorlev on Thursday January 25, 2018 @08:48PM (#16236431)
Disclaimer: I work for Google, but not on gRPC.

It's worth mentioning that gRPC is actually format-agnostic. While I can't say it does the best job of this (gRPC+protobuf works best), there's precedent for using any transport in gRPC. gRPC-Java includes examples of JSON serialization and Thrift serialization.

gRPC is just the transport protocol (its self built on HTTP/2). That said, I'd highly recommend protobufs, they're great. Define your schema in an agnostic way such that it can be easily browsed and compiled to multiple languages.

Re:gRPC-Go Engineering Practices by ocdtrekkie on Friday January 26, 2018 @08:20AM (#16238889)
As a note, a disclaimer like this given without explanation could seem to indicate gRPC is a Google product even though apparently it no longer is. I actually had a conversation with the gRPC mailing list a few weeks back about how weirdly this has been communicated on their website as well. The linked blog post continues to refer to 'we' as 'Google'.

(For those curious, IIRC it is now been handed off to a subgroup of the Linux Foundation, called the Cloud Native Computing Foundation. The change does not appear to be well advertised on their website, which contains no reference to it.)

Re:gRPC-Go Engineering Practices by mehrdada on Friday January 26, 2018 @04:45PM (#16243145)
Thanks for the feedback; we updated the footer: https://github.com/grpc/grpc.github.io/pull/622/files

Note that this is an open-source project, and the website itself is on GitHub Pages, so feel free to send us pull requests. Sometimes the reason behind things not changing is not a huge conspiracy, but no one actually having spare time to do it. :)

Re:gRPC-Go Engineering Practices by ocdtrekkie on Friday January 26, 2018 @05:18PM (#16243362)
Perfect. It'll be nice when Google isn't the only 'author' ;) but now I can clearly end up finding the CNCF this way. It just seemed apparent from the parent comment that it was possible even Googlers commenting were unaware gRPC was now under the CNCF! :)

I generally wouldn't file a PR on someone's copyright line or similar legalese, I don't know what the impact would be for any given organization or how they need to format it.

Re:gRPC-Go Engineering Practices by mehrdada on Friday January 26, 2018 @06:41PM (#16243816)
CONTRIBUTING.md in the repository[1] explicitly suggests authors to add their name to the AUTHORS file on their first substantial commit should they like to. A reviewer would work with them on the order and formatting. One thing is for sure: there is a diverse contributor base; you can see the full list in the git history.

[1] https://github.com/grpc/grpc/blob/master/CONTRIBUTING.md

Re:gRPC-Go Engineering Practices by malkia on Friday January 26, 2018 @02:48AM (#16237601)
The schema is not stored, rather think of it this way, a .proto file once it generates an interface for specific language, then this actual code understands the schema.

Now on the wire, you simply sent field numbers (e.g. the numbers that you have to manually specify in monotonically increasing way, and never reuse smaller number), and either simple type (int, float) or symbol reference to another.

But you never send the actual schema... Now some specific storage formats, might have a duplicate version of that schema, but this may be just part of their design.

So the nice thing, is that if you follow some simple rules, like: never reuse previous number, be careful when changing types of existing fields (there are only so and so ways it can go), then you can upgrade independently your services, and the data being pushed.

For example if you've added a new field (and new number), then your existing code (that's still compiled with the older schema), might just ignore it, since, even that it's there, there is no endpoint (e.g. method call to read it) for it to be accessed.

Off course, it's not so simple. After all, there are tricks, should I simply pass through fields that I do not understand to other services, or should I filter them? I certainly don't know.

I wish protos are used more and more, and ways to put them into various databases (like mysql, postgres, sqlite, etc.) is done. Then also formats that store such data in more optimal way, by rearranging such that compression can be gained, and later faster retrieval, etc.

Re:gRPC-Go Engineering Practices by cube2222 on Thursday January 25, 2018 @05:05PM (#16235136)
It's actually protobuf which is both a serialization format and data definition language. gRPC is more of a service definition language in that sense.
Re:gRPC-Go Engineering Practices by jdc0589 on Thursday January 25, 2018 @04:48PM (#16235015)
gRPC uses protobuf, they are not synonymous.