I agree the book itself was pretty half baked, but I suspect the thesis is correct. He's not claiming that crowds are smarter than individuals all the time, only when four specific conditions are met. He relies too much on anecdotal evidence and not enough on the academic research, but in the end I found the core of the argument convincing, if not particularly well expressed. Maybe I'm not skeptical enough, I dunno.
What would be interesting, would be to see some empirical research on a fascinating topic like this.