The vast majority of people you actually work with are competent, sure. But the problem with interviewing is that it adversely selects for incompetent people, because competent people are more likely to have jobs and not be currently interviewing.