Historically professors have tenure (admittedly less so now, I believe), so how they score on evaluations has no real bearing on them keeping their jobs.

I had a professor who was a terrible lecturer, and he knew his student evaluations were terrible every time. As he handed out the evaluations he would tell us that he did not read them, and did not care what we wrote.

I graduated twenty years ago, but just checked the department's website. He's still employed!

Fair enough, but even the tenured professors were untenured at some point before so they did matter and were not problematic enough to deny tenure. And tenure doesn't really apply the same for adjuncts or some lecturers, who are perhaps even more dependent on student evals. And of course it depends on the seriousness the university takes teaching, as there are R1s that are more research oriented and care less about teaching, but they account for 1% of all universities.

Even as a grad student, my teaching evaluations couldn't negatively affect me. The only people it could have real, negative consequences for were instructors, who work on contracts. As an example, my department head (perhaps unintentionally) sent everyone an email saying any scores of less than 3.73 on a 1-7 scale were grounds for termination of that instructor's contract. I did not envy them at all, basically being highly educated customer service workers.

Every large-scale student evaluation I have ever seen has been massively uninformative because students on different courses usually have drastically different expectations.

At my university, the law school was top-notch (at least, it is the best in the country) and it got the lowest score by far every year. Why? High expectations, huge number of students competing hard. The best school? Theology. Why? Low expectations, not many students in a far less competitive environment (And I studied in both and in probably four or five faculties across the school...Law was the best, Theology the worst...not even close, the former had people from Law firms, you got full feedback on everything...the latter, you usually got no feedback on any work).

This is really the point...now, you can do some kind of grouped mean model or you can compare over time...but what is the actual point? The reason they do this (or did this at my uni) was to compare departments/courses...but everyone knew it was bullshit...people lost jobs (I actually met the Dean who masterminded this, he came from the private sector, taught a minor course in the business school, and was utterly clueless).

Course evaluations have no meaningful relationship to actual learning: https://www.sciencedirect.com/science/article/abs/pii/S01914...

The usual argument that has been given in favour of evaluations is to show a correlation with final exam scores. The two problems being (1) small samples and (2) soft-ball exams leading to higher evaluation scores.

Humans are not able to accurately evaluate their own learning. When we experience fluency or are entertained, we substitute those attributes for the attribute of ability to recognise or recall information.

Conventional wisdom is that student course evaluations are primarily driven by the grades that students expect to receive. There is strong pressure to give out easy grades. I overheard two professors who were teaching different sections of the same course, get into a shouting match over grades. One of them was up for tenure, and wanted to give easy grades. He was quite explicit about this.