Facebook and Microsoft introduce ecosystem for interchangeable AI frameworks

Re:Facebook and Microsoft introduce ecosystem for interchang... by haberman on Thursday September 07, 2017 @07:07PM (#15196609)

Hi there, I work on the protobuf team at Google. A friendly note that protobuf 3.0 and proto3 are actually two separate things (this is admittedly confusing). Protobuf 3.0 was a release that added support for proto3 schemas, but Protobuf 3.0 continues to support proto2 schemas.

proto3 was designed to be a simplification of proto2 schemas. It removed several features such as extensions and field presence. Each .proto file specifies whether it is a proto2 or proto3 schema.

Protobuf 3.0 added several features such as maps, but these features were added to both proto2 and proto3 schemas.

Sorry this is kind of confusing.

[ Parent | on HN ]

Re:Facebook and Microsoft introduce ecosystem for interchang... by jiayq84 on Thursday September 07, 2017 @07:15PM (#15196651)

Thanks so much @haberman! Yep, the whole thing is a little bit confusing... We basically focused on two things:

- not using extensions, as 3 does not support it - not using map, as 2 does not support it

and we basically landed on restricting ourselves to use the common denominator among all post-2.5 versions. Wondering if this sounds reasonable to you - always great to hear the original author's advice.

Plus, I am wondering if there are recommended ways of reducing protobuf runtime size, we use protobuf-lite but if there are any further wins it would be definitely nice for memory and space constrained problems.

[ Parent | on HN ]

Re:Facebook and Microsoft introduce ecosystem for interchang... by haberman on Thursday September 07, 2017 @07:30PM (#15196721)

Hmm, I'm not sure I quite get your strategy. If you want to support back to Protobuf 2.5, then you'll need to use proto2 schemas (https://developers.google.com/protocol-buffers/docs/proto). Proto2 schemas will support extensions forever (even after Protobuf 3.0), so there's no need to avoid extensions.

You are only using proto3 schemas if you start your .proto file with 'syntax = "proto3;"': https://developers.google.com/protocol-buffers/docs/proto3 But if you do this, your .proto file will no longer work with Protobuf <3.0.

We have been working on some optimizations to help the linker strip generated code where possible. I recommend compiling with -ffunction-sections/-fdata-sections if you aren't already, and -gc-sections on your linker invocation.

[ Parent | on HN ]

Re:Facebook and Microsoft introduce ecosystem for interchang... by jiayq84 on Thursday September 07, 2017 @08:41PM (#15197075)

So what we do is to keep syntax=proto2, but allow users to compile with both protobuf 2.x and protobuf 3.x libraries. Minumum need is 2.6.1. We kind of feel that this gives maximum flexibility for people who have already chosen a protobuf library version.

[ Parent | on HN ]

Re:Facebook and Microsoft introduce ecosystem for interchang... by haberman on Thursday September 07, 2017 @10:23PM (#15197452)

Makes sense! All I'm saying is that there is no need to avoid using extensions if that is your strategy. Extensions will work in all versions of the library you wish to support.

[ Parent | on HN ]