Furthermore, I did file my taxes as an employee once, and got some €80 back, so I considered it a waste of time.
My student friends at the time swore that I should do this, because of the 1k€ "Werbungskosten", and they also got about €80 out of it, but they were earning far less, so I don't know how the maths behind that is all supposed to work out.
I'm in an entirely different situation now, I run my own company, but no employees, and have yet to talk to a tax advisor. I'll know more by the end of the year, but trust me that there is some (in my subjective opinion, of course, objective) truth in the sentence, that "rich and poor are segregated by tax systems" and this is no coincidence.
If you have the same income each month (and there haven't been changes in law during the year) each monthly payment covers exactly 1/12 of your income tax. So in that case, the December payment is the same.
Do to different circumstances you might still get small sums back, as you noticed
For most people, the withhholdings match exactly the actual income tax or the December deductions are automatically adjusted so that they are (Lohnsteuerjahresausgleich) so there is no benefit in filing taxes.
Of course the system could be much better but I don't agree with the assessment that they keep money from the poor either purposefully or by negligence.