They are specified in the contract. If you sign the mortgage contract you are giving the bank steady business for the next 10 years in return of a lower rate. By breaking that 10 year agreement the bank loses money which they will charge you for.

If the early payment penalty is based on the current rate, then it's not what I mean when I say "specified by the contract".

It all depends on the contract. You can get a not-so-low rate that gives you the option of prepayment (with some fixed penalty).