Re:Saving Millions by Dumping Java Serialization
by quest88 on Wednesday April 12, 2017 @02:55PM
(#14100654)
I'd like to know this too. As a passerby, those seem to have solved serialization, so I'm curious why you need rowfiles instead of e.g. protobuf.
[ Parent
| on HN ]
Re:Saving Millions by Dumping Java Serialization
by hrshtr on Wednesday April 12, 2017 @03:03PM
(#14100733)
One reason on top of my head: Using such communication protocol would require changes to the other services consuming it.
Re:Saving Millions by Dumping Java Serialization
by rst on Wednesday April 12, 2017 @03:25PM
(#14100926)
So did switching to their homebrew serialization format -- in fact, most of the article is about how they managed the changes (which touched codebases at multiple sites in a fairly large organization).
Re:Saving Millions by Dumping Java Serialization
by jnewhouse on Wednesday April 12, 2017 @04:06PM
(#14101250)
Those switches all occurred at the pipeline level, leaving the map-reduce platform untouched. Switching our base logs to something like Parquet, Thrift or Protobuf would be a much larger project. We do support writing and reading Parquet to allow us to interface with other big data systems.