News for hackers. Stuff that matters.
Re:Yes Prime Minister, questionnaire design matters by arp242 on Tuesday February 20, 2024 @02:35PM (#39445824)
I don't agree with the conclusion; 3% difference is basically within the error margins for these types of things. I typically assume a ~5% error margin, which is nice round number I just pulled out of my ass, but it seems roughly accurate.
Re:Yes Prime Minister, questionnaire design matters by quatrefoil on Tuesday February 20, 2024 @02:38PM (#39445860)
You're misreading the answers. In sample A, 45% are in favor of reintroducing National Service and (38% oppose it). In sample B, the phrasing is reversed, meaning that 34% are in favor (and 48% oppose it). So, the difference is 11 percentage points.
Re:Yes Prime Minister, questionnaire design matters by arp242 on Tuesday February 20, 2024 @02:53PM (#39446065)
Eh right, do'h >_<

But in that case you need to add a control for both questions without the prior four questions. How do you know the difference in phrasing doesn't matter?

Re:Yes Prime Minister, questionnaire design matters by IshKebab on Tuesday February 20, 2024 @02:48PM (#39445994)
> (38% oppose it)

No, you're misreading too - see my other comment.

Re:Yes Prime Minister, questionnaire design matters by cool_dude85 on Tuesday February 20, 2024 @02:36PM (#39445843)
Re-read the question. The first one is 45-favor, 38-oppose, the second one is 48-oppose, 34-favor.
Re:Yes Prime Minister, questionnaire design matters by IshKebab on Tuesday February 20, 2024 @02:45PM (#39445955)
No it isn't. The first one is 45-favour, 38 don't favour. That's a totally different thing.

This proves the idea that you can get a result that points the way you want it to. But it doesn't quite prove that the leading questions have any effect.

To do that they would need to ask 2 more sets of questions with the "oppose"/"support" questions inverted. Otherwise the effect could just be due to the different wording of the final question.