Actually it does not drive down the price but the yield. When the price goes up the fixed payment is a lower fraction of the price of the bond. Hence the return goes down.

https://www.investopedia.com/terms/b/bond-yield.asp