News for hackers. Stuff that matters.
Re:Evidence of fraud in an influential field experiment abou... by civilized on Tuesday August 17, 2021 @09:56PM (#28216967)
Dan Ariely's response bothers me. Here's the crucial passage:

> The work was conducted over ten years ago by an insurance company with whom I partnered on this study. The data were collected, entered, merged and anonymized by the company and then sent to me. This was the data file that was used for the analysis and then shared publicly.

On a plain reading, Ariely clearly states here that the data file prepared by the insurance company was the same as the one used for the analysis and shared publicly. But we already know that he modified the file:

- He is listed as the creator and last modifier of the published Excel file.

- In an email to his collaborator Nina, he admitted that he switched the outcome labels in an attempt to make the data easier to understand. So clearly the published file was not identical to the one provided by the insurance company.

So I worry that Ariely is taking us for a ride here. He had the most incentive to get the desired results, far more than the nameless insurance company.

The original insurance company dataset, if it exists, is probably lying around in an email somewhere. They were, after all, able to provide the final version, which was prepared not too long after the original.

Re:Evidence of fraud in an influential field experiment abou... by woofie11 on Tuesday August 17, 2021 @10:35PM (#28217211)
He indubitably is. And this is common. MIT uses NDAs and non-disparage agreements to cover stuff like this up, unless you bring it up first in a very public forum. Stanford has researchers who are well-known outright frauds now driving education reform in California. I know of incidents at Brown too.

This stuff is common, especially in elite schools, and especially among researchers with a lot of popular uptake. Stuff like this brings academic jobs.

Universities have no incentive to fire high-impact professors, and internal channels are populated by friends.

Competition for faculty positions is 1000:1 at elitist schools; jobs aren't available for those who cheat.

Re:Evidence of fraud in an influential field experiment abou... by radicaldreamer on Wednesday August 18, 2021 @02:50AM (#28218314)
Could you elaborate on the Stanford education reform topic?
Re:Evidence of fraud in an influential field experiment abou... by woofie11 on Wednesday August 18, 2021 @10:01AM (#28221013)
Do a quick web search for Dweck:

1) Results don't replicate.

2) Look up her article in Nature, which claims to be preregistered, then look at the actual preregistration. See the data mismatch.

3) Read her book, and look at the claims in the introduction. Compare those to effect sizes. Are the claims supported or contradicted even by her (already baked) data? By replication studies?

Read the Boaler books and studies. You'll find a lot of fake information too. Not just studies, but also historical stuff (e.g. passages about Einstein and other famous folks).

This is kind of an open secret in the ed research community. Everyone knows about it. Few dare talk in public. You can see outcomes for those who did. And for the general public, it's too technical. That a preregistration comes before data collection isn't something one can explain to a popular audience.

This is the culture at MIT, Brown, Stanford, and I believe many other elite schools. Those three are just where I have first-hand information.

Re:Evidence of fraud in an influential field experiment abou... by tonystubblebine on Sunday August 22, 2021 @11:59PM (#28272201)
Dweck's a good example too of how the research and/or the media coverage can misunderstand the results. I think it's fair to say that the growth mindset effect is understood as having come from knowledge, i.e. you teach someone about growth mindset and then their performance improves. But my mom, a teacher, tried to apply that idea to her class and the change actually came down to habit. She made her class institute a self-talk habit related to growth mindset and then she reinforced that habit for an entire year. That's an anecdote, not hard science, but it certainly seems like a much more sensible pathway between the concept of growth mindset and then improved results. I wrote up her protocol, which is for third graders, here: https://betterhumans.pub/growth-mindset-is-a-habit-you-want-...
Re:Evidence of fraud in an influential field experiment abou... by bambax on Saturday August 21, 2021 @12:50PM (#28258456)
This really does not sound good.

> he said that all his contacts at the insurer had left and that none of them remembered what happened, either

At other times he is quoted as saying that he "doesn't remember", which does remind one of politicians or other guilty parties saying they "don't recall" any of the many remarkable details of past shenanigans.

He seems to be guilty as hell.

Re:Evidence of fraud in an influential field experiment abou... by frankster on Wednesday August 18, 2021 @03:19PM (#28225901)
His statement is written in two different fonts! His name is written in a different font to the rest of the statement.

This doesn't prove much, but it's an embarrassing coincidence for him.