The best balance I’ve seen involves centralizing a small number of essential services, ideally ones with lots of compliance and security complexity. Manage that well in one place, then let the departments use that infrastructure to meet their unique needs.
Face it - students have higher expectations now, professors also have higher expectations. This requires administrative staff to run. Back in the day school budgets were lower, but even when I went to college in 2005 they didn't have campus-wide wifi in every classroom. We had one professor who taught with powerpoint. Today, every student has a laptop in class.
Maintaining a modern campus takes a big IT department and centralizing it is the least wasteful way to do things.
There are things better done by a central IT team like university level WiFi, but you can make that smaller and also have departmental teams for things where more agility is needed. If the people are competent it's really great.
And yes 3-4 people only makes sense because it was a large department, but smaller departments with similar mandates, for example English/Literature and History, just have a shared departmental IT between them.
For instance, I can tell you right now with certainty that at any large university the number of software devs or database admins in the IT department far outpace the number of financial analysts working in foundation/endowment. Pick any large university at random, and I'll wager that without even knowing the spread.
But here's the thing, universities need IT divisions. They also need the other large operations level bureaucracies they typically have put in place. Facilities and plant, university police, housing, etc etc. You can't pull off a large university without these divisions nowadays. So saying, "Oh we can cut them" is very shortsighted.