When managing millions of transactions through Kafka that have to be serialized to s3, read by hive natively, translated to parquet by spark, support nullable complex types (map, struct, array), allow schema evolution forward and back, then Avro is a good choice for your base data format.