Binary will always beat the pants off human-readable formats. I doubt Excel can export to Flatbuffer, Protobuf, MessagePack, Parquet, etc. So CSV is pretty much here to stay, even though correctly parsing it is more difficult that it seems.

To gain even from parallelism, make all your rows a fixed size. Arbitrary length fields aren't your friend if you want max speed.