An HR department is not a "big company" thing that's there to get in your way. Its job is to do things like (e.g.) protect the company against liability in messy sexual harassment scenarios or (e.g.) hiring discrimination lawsuits.
The substitute for this functionality is not goodwill or "just doing the right thing" because this role necessarily involves a detailed understanding of employment law. If you disregard this department when building your company past, say, tens of employees, you are basically certain to be in for a bad time later. The book is written here by companies like GitHub, who have had major scandals that could have been prevented mostly or wholly by a competent HR department. If you look at their example and don't learn, then you are a fool.
Beyond filing paperwork, I agree that hiring shouldn't be an HR job. Unless HR is hiring for HR, let the teams decide who to hire.
Often they hire based on just that. If they don't, step two is usually some sort of assessment, and the assessments I've done tend to be pretty good.
It is possible that I've forgotten a ton of interviews with crappy companies that I didn't end up working for. The interviews I remember are the ones that worked.