Big Five is standard in psychology. It should not be confused with personality quizzes on the internet. Big Five scores are reproducible (two tests at different time give similar score) and predictive (correlate with outcomes of interest). Its place in psychological metrics is second only to IQ.

I don't dispute that it is measurable or reproducible, just that it is meaningful. The problem with self-report is that you can only ever measure perception.

Let's say I want to measure ability in chess. I give everyone a self report survey: how good are you at chess, 1 to 10? Group A averages a 6, Group B averages a 5. Reproduced across time and regions. Is group a better at chess? We have no idea. All we know is that they report a high average than group b.

Switch to big five: "I feel comfortable around people." Rate the accuracy. Even if group a rates more accurate than group b, what do we really known from this? Comfortable is a relative term. Do group a and b have different perceptions or different experiences?

If your self-report chess score correlates with chess winrate, you'd admit it's meaningful.

It's same for Big Five. Self-reported neuroticism correlates with, say, diagnosis of depression. It's not just perception because it correlates with outcomes of interest.

Well yea, if perception of chess skill corresponds with actual chess skill then it means people are capable of self evaluating chess skill.

But that's not your example. If your example was self reported depression correlates with clinically diagnosed depression, we now know that people can self-perceive depression the same as clinicians diagnose it.

Your example is self-reported neuroticism corresponds with depression. So what? How are we any closer to knowing if we measured neuroticism? Is there a pre-existing link between neuroticism and depression? How did they measure neuroticism to establish that link?

It's funny how one day Hacker News has a scathing discussion on The Crisis of Reproducibility in Psychology, and the next, it's commenters will fight to the death to defend it.

Contrary to what you believe, there is no contradiction here. Implicit association test and stereotype threat (to name two large offenders) had reproducibility crisis. IQ and Big Five findings replicated a lot and basically have no reproducibility problem.

Pieces of psychology can have reproducability problems, without the complete & total entirety of the field having reproducability problems.