Maybe it’s like this: Given that I have no experience with Go, I’d still rate myself at least 3 when someone asks me how well I’d be able to work with it because the job they’re asking it for is ‘back-end development’.
Of course this falls apart the moment they ask me any Go question during the interview, but not when I’m actually asked to build a web application in it after being hired.
(the one time this happened people thought I’d been working with the new language for years, but no, it’s just a whole lot of transferrable skills)
The self-rating is useful because it gives you a measure of their perceived proficiency in skills relative to each other. You still have to ask a question like you did, but now that you know that "3/5 stars in Node" means advanced beginner level, you have a good idea what to expect when they rate their SQL skills 5/5 or their CSS 2/5.
Sorry: I misunderstood slightly, thinking it was something you put to them. In our tech interviewing I ignore self-assessed ratings, finding them useless for the reason you do, but I don't hold it against them if they're inaccurate just for the reasons I outlined.
I feel like you guys think I asked them to rate themselves 1-5 on the subject. I did not, I got their resume, where they themselves rated them on the subject. With my question I was just checking whether that self-assessment was correct. I don't see how that is a bad question... Should I just believe their resume? Should I give them some programming assignments? Should I ask them to write a complex algo on a white board? I feel like just asking is a good way, personally, but maybe I'm wrong.