Re:Yes Prime Minister, questionnaire design matters
by arp242 on Tuesday February 20, 2024 @02:35PM
(#39445824)
I don't agree with the conclusion; 3% difference is basically within the error margins for these types of things. I typically assume a ~5% error margin, which is nice round number I just pulled out of my ass, but it seems roughly accurate.
[ Parent
| on HN ]
Re:Yes Prime Minister, questionnaire design matters
by quatrefoil on Tuesday February 20, 2024 @02:38PM
(#39445860)
You're misreading the answers. In sample A, 45% are in favor of reintroducing National Service and (38% oppose it). In sample B, the phrasing is reversed, meaning that 34% are in favor (and 48% oppose it). So, the difference is 11 percentage points.
Re:Yes Prime Minister, questionnaire design matters
by arp242 on Tuesday February 20, 2024 @02:53PM
(#39446065)
Eh right, do'h >_<
But in that case you need to add a control for both questions without the prior four questions. How do you know the difference in phrasing doesn't matter?
Re:Yes Prime Minister, questionnaire design matters
by IshKebab on Tuesday February 20, 2024 @02:48PM
(#39445994)
> (38% oppose it)
No, you're misreading too - see my other comment.
Re:Yes Prime Minister, questionnaire design matters
by cool_dude85 on Tuesday February 20, 2024 @02:36PM
(#39445843)
Re-read the question. The first one is 45-favor, 38-oppose, the second one is 48-oppose, 34-favor.
Re:Yes Prime Minister, questionnaire design matters
by IshKebab on Tuesday February 20, 2024 @02:45PM
(#39445955)
No it isn't. The first one is 45-favour, 38 don't favour. That's a totally different thing.
This proves the idea that you can get a result that points the way you want it to. But it doesn't quite prove that the leading questions have any effect.
To do that they would need to ask 2 more sets of questions with the "oppose"/"support" questions inverted. Otherwise the effect could just be due to the different wording of the final question.