There fact that is not a consequence of statistics is that we observe the universe to not be in the state of maximal entropy. That is sufficient to explain asymmetry of time, I think. The mystery is -- why wasn't the universe in a state of maximal entropy at time 0?

The anthropic principle requires any self-reflective universe to not be in a state of maximum entropy. If our universe where in a state of maximum entropy, there would be no one around to ask.

Elon Musk hadn't started his simulation yet /s

That's one of the fundamental questions of physics. Hawking's Cambridge lectures audiobook explicitly cover the topic of what's up with the second law being statistical, why the arrow of time is apparently directional, and possible explanations for how you can answer those questions and explain the existence of the universe without requiring a "cause" for the big bang.

Look at every possible two particle collision. To simplify, consider one of the particles as the initial frame of refernce so you now just have impact angle, which is random.

Now play back time by looking at that particle's post collision path as the frame of reference and play time backwards. You will find a bias in input angles for these collisions.

Universe started from low-entropy state. From the big bang, there is nowhere go but downhill.

In a microscopic level the physics can be time-symmetric (it seems not to be the case, there are T-symmetry violations), but macroscopically universe had higher entropy in the past (cosmological arrow of time)

edit: CPT-symmetry -> T-symmetry .

Which experiment saw CPT violation? CPT is a straightforward consequence of QFT with minimal assumptions.

Sorry, I meant T-symmetry violation, corrected.

Kaon decay and B mesons decay break CP-symmetry and a CP-symmetry violation is equivalent T symmetry violation (CPT symmetry is preserved only if CP-violation is paired with T-symmetry violation)

You are right in that we need to assume that the universe started in an extremely small entropy state. This is basically the past hypothesis. (Which can be justified to some extent using anthropic principle and eternal inflation, however these two ideas are very controversial among physicists.)