Fair has nothing to do with it. Contracts don't need to be fair, and often aren't. They just need consideration. If we sign a contract whereby you give me your car, bank accounts, and house, for $1, that's a valid contract.

The only part which wouldn't be valid in a contract was the first-born child. That was a joke.

Indeed, if the GPL were a contract, courts might compel compliance.

However, the GPL is not a contract, it's a license. The FSF bent over backwards to make sure the GPL/AGPL licenses wouldn't be viewed as a contract, in part to limit liability / damages / risk.

Confusingly, some EULAs are framed contracts, contrary to the acronym, and do expose users to much more risk of liability than the GPL.

The relevant part of the GPL is:

    You are not required to accept this License in order to receive or
    run a copy of the Program. Ancillary propagation of a covered work 
    occurring solely as a consequence of using peer-to-peer transmission to 
    receive a copy likewise does not require acceptance. However, nothing 
    other than this License grants you permission to propagate or modify 
    any covered work. These actions infringe copyright if you do not accept 
    this License. Therefore, by modifying or propagating a covered work, you 
    indicate your acceptance of this License to do so.

Although we often like to take a plain-text read, but that's misleading; this is legal jargon. It's one of those bits of text which needs to be explained by a lawyer, and one who specializes in both licensing and in contract law.