I guess it was good before compression. Nowdays working with protobufs is painful inside Google as well, but at least it's supported by everything.

Most of what the CPUs at Google are doing is just copying fields from one protobuffer to another.

I think that's a fair description of pretty much every webapp :D

Most of them are doing nothing more than copying data from the database into an http stream.

Well, all what CPUs do is copying memory from one memory location to another, perform some arithmetic and do some conditional jumps :-)

Most of the arithmetic is checking optional fields if they are empty or not before copying :)

pop pop jump jump oh what a crunch it is.

(sung to this tune https://www.youtube.com/watch?v=iENQXIQ8wH0 )

Side note: It really is incredible what happened in the early days of computing when memory and computation were limited. How much care was taken in the precise layout of memory or even the timing of a calculation was insane.

Don't forget the validation!

Webapps are dumb middleware that pipes data from the database into an http stream - but it needs to determine which database calls to invoke and sanitize all the incoming junk.

We use arrow to do the validation 'for free': it has a typed columnar schema, so by passing the schema, we get the recordset validation without inspecting values.

we still need to do semantic validation, but w/ arrow, now we do that in bulk and on gpus :)

Sure, but when you are using C++, protobufs are not the best way to store data...but I guess it could be worse.

At least when I was there, proto (de)serialization consumed the plurality of cpu cycles, but not the majority.

It isn't really the majority these days, is it?

You're right, it's probably fair to say it's a stunningly bad tradeoff for most applications most of the time, given we have fast compression like snappy & brotli available now and cloud costs are heavily weighted towards CPU costing more than storage & network transfer