That still feels sleazy and racketish. I can understand doing that when you take on a new customer, but charging you thousands of dollars just to change the exact same loan to one with a different rate? [1]

Reminds me of the time I bought a package at a pawn shop, but didn't want one of the items in it. They said that to get a discount, I'd have to buy the whole thing as is then pawn back the unwanted item. So far, so good, but then I had to give my ID and attest that I didn't steal that one item ... even though they knew it never came from me to begin with!

[1] The way you said it, thousands sound like the typical case and $500 is a special deal.

If you do a full re-fi; the new lender is going to run your credit, do an appraisal of your home, record the new loan on the title, do a title search, purchase title insurance, pay the broker's commission, etc; that all costs money, and that's where the thousands come from.

Just adjusting the rate in their systems certainly doesn't cost the lender nearly $500 or $1000, but it was still a win-win. They got some money to offset the lower rate, and got to keep servicing the loan, and I got to pay less interest, it's been a while, but I seem to recall my break even was about 3 years each time. I would certainly consider the availability and price of rate modification when considering lenders in the future.

So since it seems like under this system, the lender carries most of the downside risk on interest rate movements while the borrower gets the upside, the lenders must be covering this with a greater spread between their cost of borrowing and the fixed interest rates they charge?

Usually yes; the rate for fixed mortgages tend to be higher than adjustable. Looking at rates today, I'm seeing about 3.75% for a 30-year fixed; 3.125% for 15-year fixed, and 4.25% for a 5/1 ARM; but it's more typical in my experience for an ARM to come in near or below the 15-year fixed rates.