The problem is not "oh noes people are paying us back" it's "well crap, the private sector is undercutting our legislatively defined interest rates and we will be vastly under target on these returns."
If the government never "had" the money (again, in the accounting sense, loan payments are pretty stable investments normally), then they can't lose it (as in your counter-point). However, if they did "have" the future money (in the accounting sense), and they can no longer expect that to come in, they are in fact losing money (again, on the books). In these cases, there's no actual dollar bills flying out of the window, just some accounting projection spreadsheets somewhere that make people unhappy.
But what I'm really objecting to is the conclusion that the future loss of interest represents a loss of income, when they now have the principal back and can invest that in the currently best way possible. If what you get from those funds today is significantly worse than the projected income from those loans were, then it seems you are implying that student loans really are meant to shaft the people who take them out.