From watching two seasons of The Grand Tour there also seems to be a clause in the contracts similar to "Jeremy and Andy get to do anything they want" - which probably reduces the risk of someone being punched dramatically.