I sometimes run into someone who has a lot of trouble speaking English and it is honestly frustrating. If English is their first language then that is a bit of a red flag. However, in cases of developers who speak English as a second or third language, I've often found it often has nothing to do with their intelligence or competence. Some of the fastest learners and most promising people I've worked with started out barely speaking English.

Is it fair to require good communication skills on an interview? I would probably say so. However, you may miss out on some excellent candidates. Best of luck in your hiring!