> It is worth noting that many gladiators, and Olympic competitors, were in fact not slaves and many were actually quite wealthy.

Gladiators and Olympic competitors are completely different things. Olympic competitors were free men - I don't think slaves where allowed to participate.

Gladiators on the other hand were mostly slaves. It was considered a dishonorable occupation - basically on the level of prostitution - and therefore not suitable for a free man. If a free man partook in gladiatorial combat he would lose a number of civil rights.

Chariots were owned by wealthy free men, and presumably they would be the ones paying a fine.