News for hackers. Stuff that matters.
Re:Star neuroscientist may have manipulated data to support ... by llm_nerd on Friday December 01, 2023 @09:19AM (#38486939)
>"When you only have 2-5 people that _actually_ read a paper"

Even when a paper goes through a vigilant, rigorous peer review, it relies upon the data that the research team supplied. Over and over again cases like this have encountered manipulated data. Humans are flawed creatures, and if you spent a lot of time and professional credibility on a hypothesis, there is a strong motive to find what you sought. Doubly so if the results are salacious or contrarian in some way and thus get viral attention. Just convince yourself that it's the data that was wrong somehow and you know your assumptions are right so just this One Time you need to do a little manipulation.

A study I have seen cited on here countless times is the "honesty pledge" one by Ariely et al. It was the one that claimed that when a person signs a form at the beginning, they're more honest. It was complete and utter BS, based entirely on fabricated data. It joins an infamous list of studies that have had enormous influence (especially if they have an "aha!" factor -- if it is the sort of thing that Malcolm Gladwell would talk about, consider it suspect) but were the creation of someone making up data in Excel.

Re:Star neuroscientist may have manipulated data to support ... by bumby on Friday December 01, 2023 @09:36AM (#38487132)
If I'm not mistaken, Ariely is asked about this on the "Armchair Expert" podcast. His claim was that he did not manipulate the data personally and it was someone further upstream whom he trusted. His point was that at some point, trust has to enter the process, except in the rare cases where a single person is doing all the research themselves (apologies if I'm misremembering this, but I think the following point still stands.)

IMO the researcher still has some responsibility because ultimately, it's their research. So the questions to me are:

1) How much due diligence is reasonable? Does it change depending on the source? For example, is it more/less reasonable to accept government-provided data at face value vs. data collected by an undergraduate?

2) What processes can be implemented to safeguard data manipulation? I know there is a movement to provide data with peer-reviewed submittals, but it's still a low probability that a peer-reviewer has the time or inclination to really dive into the data to assess the claims.

Re:Star neuroscientist may have manipulated data to support ... by nvm0n2 on Friday December 01, 2023 @03:45PM (#38492116)
Ariely tried to frame both his assistant and the insurance company he got the data from. The insurers provided the same data to a journalist who found there had been very substantial alterations made. The assistant showed that the Excel metadata indicated Ariely was the last to edit it.
Re:Star neuroscientist may have manipulated data to support ... by bumby on Friday December 01, 2023 @07:04PM (#38494329)
Yeah, I read the same digging a bit more after I made that original comment. It’s much more damning than what he made it sound like in the interview, although it didn’t say how substantial his edits were.
Re:Star neuroscientist may have manipulated data to support ... by nvm0n2 on Saturday December 02, 2023 @05:55AM (#38497676)
Very substantial, to the extent that the dataset was made up. The original data obtained from the insurer showed no effect.

The more interesting thing though is why he chose to investigate this question in the first place and why he chose to do fraud to make it seem true. The hypothesis is a very weird one and there's no reason to think it would hold. Unless that is you think of people as being child-like lumps of Playdough, so easily manipulated that trivialities like where exactly something appears on a form can yield huge behavioural differences.

That belief is the only reason you'd ever come up with such a hypothesis, and I think it's not really surprising that someone like that would engage in fraud. After all they have spent months (or years?) on trying to prove that people's levels of honesty are trivially controlled by psychologists like yourself. If you believe that's true then why wouldn't you commit fraud? After all you can easily manipulate people into not noticing it.

Re:Star neuroscientist may have manipulated data to support ... by bumby on Saturday December 02, 2023 @11:01AM (#38499504)
The manipulation seems substantial, but the point I was alluding to was that there wasn’t a smoking gun (at least by the amounts that I’ve heard and read) that Ariely made those substantial changes. I’m not sure what was included in Excel metadata, but it’s at least conceivable that the copy/paste + random change edits were done by someone else prior to Ariely edits. Where it gets damning is that he was the one the original dataset was sent to and the last to edit it. At the very least, it shows a lack of due diligence in not catching there were somehow many more datapoints added.

Just conjecture of course, but this came at a time when governmental “nudges” were very en vogue. I could see where successful research could be thought of AAA a pathway to influence, prestige, and money through government grants and appointments. And there were some highly regarded behavioral psychologists who were substantiating its effectiveness.

Re:Star neuroscientist may have manipulated data to support ... by nvm0n2 on Saturday December 02, 2023 @02:40PM (#38501326)
I think if the data goes insurer -> Ariely -> assistant, and both insurer and assistant present evidence that they didn't do it, then there is certainly a suspiciously large amount of smoke for there to be no fire. Short of CCTV footage showing him doing it, it's hard to get stronger evidence.

Yeah, governments love the idea that they can influence the population via simple tricks. That's understandable.

Unfortunately nudges are still very much en vogue. COVID was nothing but endless nudging, maybe more like pushing, with tricks like making everything into a social responsibility towards others being deployed endlessly even when not supported by the underlying facts. It worked extremely well. That said, I'm not sure you need psychologists to tell you that "do it for your grandmother" is a powerful manipulation tactic. A lot of the valid findings in psychology are obvious, and the non-obvious findings are often invalid. So we could just defund that field and not lose much IMHO. I say that as someone who has studied psychology. I have a good friend with a PhD in it who thinks the same.

Re:Star neuroscientist may have manipulated data to support ... by robswc on Friday December 01, 2023 @09:45AM (#38487229)
Oh for sure. It would be very easy to determine if the data is bad (in most cases) if someone tells you where to look. It is almost impossible to expect people to put their mental resources to trying to "crack" every paper that comes across their desk tho... so we have a bit of a problem, to say the least.