That's no guarantee of the rigor, of course, but I don't think yours is a fair characterization of the book. Are you sure you're not confusing it with Gladwell? ;-)
I went in really wanting to love the book, it was universally praised and I was pretty high on Ariely at the time, but even with that bias I still found large chunks quite unconvincing.
Perhaps it was just the writing style or something, or the fact that the beginning of the book focused a lot on those priming studies at a time when that entire area of research was profoundly embarrassing the social sciences.
(re: Gladwell, that's a good way of putting it, it was uncomfortably Gladwellian! Not that level obviously, but not far enough away!)
My favorite part was his discussion of the Linda problem, where he would get different results depending on how the question was phrased. In the book, he never once considered that the problem may be that his formulations of the problem were not well communicated to the study participants.
In the end, I decided that his research into cognitive bias was more a journey of self exploration, but somehow he discovered these biases in everyone except himself.
Maybe fixing humans is impossible and it would be better for meta-meta-cognition to be accomplished by other people. Clearly they are quite good at it already!
Suppose you had a complete record of a person's speech and writing i.e. some of their outputs.
Could a software be devised which gave the equivalent of a 3rd party observation on your choices? Could you even identify choices from the data?
Perhaps it's that Kahneman is in the "uncanny valley" between science and popular writing. Since Gladwell is (for me) safely on the popular side, I find him engaging rather than misleading. But I'm pushed away by Kahneman's apparent overconfidence in places that I'm certain he's wrong, including such examples as Jason gives.
I find this whole "well I examined my intuitions about X so that is some evidence of this model of human decision making about X" that is immensely popular in philosophy (especially meta-ethics) and economics to just be embarrassing garbage. So that definitely hit a very strong bias I have. And it seemed completely out of place in a book concerned with human irrationality.
Weak standards of evidence also made me bit uneasy throughout. But my System 1 just trusted the whole book because everyone seems to be liking it. Case of Social Proof i guess.
Note that Kahneman is a psychologist, though he is affiliated with behaviorial economics.
I did find him to be a hell of an interesting thinker and someone with very interesting ideas, but to me, the whole book had a smell of overconfidence and incredibly weak standards of evidence. Very much in the "well I observed this phenomena, went for a walk and though deeply and clearly the reasons is X", followed by a tower of (interesting) conjecture built on top of X.
I put it down to coming from the economics culture rather than a more rigorous scientific culture.