News for hackers. Stuff that matters.
Re:No evidence for nudging after adjusting for publication bias by sacrosancty on Sunday July 24, 2022 @08:41PM (#32219353)
I think social scientists have lost the right to the benefit of the doubt. They don't preregister their trials. They don't publish their negative results. They're notoriously bad at statistics, notoriously let their political beliefs distort their conclusions, notoriously scatter their work over thousands of arbitrary, hard-to-compare small-sample-size studies instead of concentrating their resources. I don't think it matters if this criticism is right or not. The fact that it's even possible to make such a criticism is already a condemnation of the scientific incompetence of social scientists.
Re:No evidence for nudging after adjusting for publication bias by unityByFreedom on Sunday July 24, 2022 @10:32PM (#32219992)
> They don't preregister their trials.

First of all, preregistration is not a requirement for the scientific method, which has functioned well for centuries. That is a recent trend in response to the overflowing amount of haphazardly published science.

Second, it is up to the individual scientist to decide to preregister or not. Some social scientists may preregister.

Third, small sample size may be a fair critique, however that overlooks how difficult it is to collect such data.

You've made a lot of generalizations here that amount to, "social scientists aren't as rigorous as other areas of science, therefore we should only believe studies that disagree with their results". I don't think throwing the baby out with the bath water is helpful. You can take results of studies with small sample sizes with a grain of salt, watch for replication, etc. Lambasting the field as a whole doesn't make sense to me.

Finally, readers should note that this isn't a new argument. People have been making this claim about social science for 120 years, if not longer, but at least since Freud and contemporaries began publishing.

Re:No evidence for nudging after adjusting for publication bias by rayiner on Sunday July 24, 2022 @10:38PM (#32220013)
> You've made a lot of generalizations here that amount to, "social scientists aren't as rigorous as other areas of science, therefore we should only believe studies that disagree with their results".

I think it’s more “ignore them completely.” It brings “science” into disrepute to let social science associate with the other sciences.

> I don't think throwing the baby out with the bath water is helpful.

There is no baby!

> People have been making this claim about social science for 120 years, if not longer, but at least since Freud and contemporaries began publishing.

Doesn’t that prove the point? It wasn’t science then and isn’t science today.

Re:No evidence for nudging after adjusting for publication bias by unityByFreedom on Sunday July 24, 2022 @11:22PM (#32220204)
> It brings “science” into disrepute to let social science associate with the other sciences.

So you just want it renamed to "social studies" or what? What is your proposal, that nobody research this topic, or that they be separated in journals etc? I doubt that will have much impact on whether it makes the news. If you want that to change, you may need to get yourself onto the board of a journal you care about.

> There is no baby!

That's reductive. Just because you don't see the baby doesn't mean it doesn't exist.

> Doesn’t that prove the point? It wasn’t science then and isn’t science today.

No, it just proves it's an old disagreement, like nature vs nurture.

There is plenty of work in social science that contributes to humanity. It will always have smaller sample sizes due to the nature of collecting the data. The work can be considered useful nonetheless.

Re:No evidence for nudging after adjusting for publication bias by Schroedingersat on Monday July 25, 2022 @01:06AM (#32220682)
> So you just want it renamed to "social studies" or what? What is your proposal, that nobody research this topic, or that they be separated in journals etc? I doubt that will have much impact on whether it makes the news.

Or maybe people just want to shine light on the fact that social science is harder than other science for a bunch of different reasons, bring social scientists' attention towards the tools that help mitigate this, and bring the journals that seek profit over reliable results into disrepute?

Most of the papers published in current social science journals are not science, and this has been a problem for those 120 years precisely because the techniques used in chemistry or physics are inadequate for the problem domain, so applying them blindly does not produce scientific outcomes.

Re:No evidence for nudging after adjusting for publication bias by unityByFreedom on Monday July 25, 2022 @08:56AM (#32223627)
> maybe people just want to shine light on the fact that social science is harder than other science

The comment to which I was replying lacked the nuance in yours. Context is everything.

Re:No evidence for nudging after adjusting for publication bias by rayiner on Monday July 25, 2022 @02:20PM (#32228665)
> So you just want it renamed to "social studies" or what? What is your proposal, that nobody research this topic, or that they be separated in journals etc

Yes. And that the rest of us stop treating it as science, citing it as science, and relying on it as science.

For example, there is a major trend in the law of treating social sciences as having truth value the way real sciences do. That’s the kind of thing we need to stop doing.

Re:No evidence for nudging after adjusting for publication bias by unityByFreedom on Monday July 25, 2022 @09:42PM (#32233297)
I found the hard/soft science discussion here [1] to be informative.

It seems unlikely that all of social science will one day be declared as "not science". Aristotle's methods, for example, did not require a certain sample size.

You're applying far too strict of a definition to science. Basic forms of science can be practiced by a child at home. Journals publish more in-depth analyses, and it's up to them what to publish, at the risk or reward of gains and losses of readership.

[1] https://blogs.scientificamerican.com/cross-check/is-social-s...

Re:No evidence for nudging after adjusting for publication bias by rayiner on Monday July 25, 2022 @10:08PM (#32233434)
> You're applying far too strict of a definition to science.

This isn’t a fun theoretical exercise. In the public sphere, “science” tends to get invoked with dispositive weight. And it should in many cases. But for that to work, “science” must meet the level of rigor people associate with “science.” To be called “science” it should be like physics in terms of providing truth value, not psychology.

Aristotle didn’t do “science.” He was a philosopher. His ideas were precursors to science, but weren’t science.

Re:No evidence for nudging after adjusting for publication bias by unityByFreedom on Monday July 25, 2022 @10:27PM (#32233559)
> To be called “science” it should be like physics in terms of providing truth value, not psychology.

It's possible to form a truth about human behavior, for example, I did X because Y. If someone points my behavior out to me, then in the future I may do Z in response to Y. Human behavior can change upon observation [1]. That doesn't make the study of human behavior "not science" in my opinion.

> Aristotle didn’t do “science.” He was a philosopher. His ideas were precursors to science, but weren’t science.

In that case, I suppose you will acknowledge that Galileo did science, despite not having a lot of data points. I think Aristotle did too, because I draw the line at hypothesis, observation and conclusion, which may or may not result in some ultimate truth that remains constant.

I suppose you would also say that the double slit test is science. Yet that result changes, and there is no truth value that we can explain, except by noting that the result changes when the experiment is observed.

[1] https://news.ycombinator.com/item?id=32233420

Re:No evidence for nudging after adjusting for publication bias by planck01 on Tuesday July 26, 2022 @01:39AM (#32234673)
> That doesn't make the study of human behavior "not science" in my opinion.

It might be valuable and it might be worthwhile to do. But what you describe in itself is not science.

Re:No evidence for nudging after adjusting for publication bias by unityByFreedom on Tuesday July 26, 2022 @02:34AM (#32235089)
> science: the intellectual and practical activity encompassing the systematic study of the structure and behaviour of the physical and natural world through observation and experiment. [1]

Given the definition, why wouldn't you consider observing human behavior to be science?

[1] https://www.google.com/search?q=define+science

Re:No evidence for nudging after adjusting for publication bias by istjohn on Monday July 25, 2022 @12:00AM (#32220354)
It's not useful. In fact, it's actively harmful. The constant churn of findings and retractions undermines the public's belief in science. When people stop believing in science, you get things like flat earthers, global warming deniers, and vaccine skeptics.
Re:No evidence for nudging after adjusting for publication bias by unityByFreedom on Monday July 25, 2022 @12:35AM (#32220516)
The quibble here is whether social science is a science. Here's a good overview, including a video of Feynman which the HN crowd may appreciate:

https://www.quora.com/Is-social-science-a-real-science/answe...

Humanity is observable. It's just hard to collect the data. I think you can make the case that some science isn't as rigorous, or that the jury is still out, but to say that it isn't science at all is wrong in my opinion. Even Feynman acknowledges that conclusions may be drawn later.

Aristotle philosophized quite a bit and is considered an early contributor to the scientific method. More food for thought:

https://blogs.scientificamerican.com/cross-check/is-social-s...

Re:No evidence for nudging after adjusting for publication bias by istjohn on Monday July 25, 2022 @04:06PM (#32230213)
I think social science is clearly possible. The question is whether what passes for social science in our scientific journals deserve the label.
Re:No evidence for nudging after adjusting for publication bias by unityByFreedom on Monday July 25, 2022 @09:35PM (#32233261)
That is a much more reasonable question that I think would be up to the journal, and you as a reader.
Re:No evidence for nudging after adjusting for publication bias by sbierwagen on Monday July 25, 2022 @02:41PM (#32229026)
>First of all, preregistration is not a requirement for the scientific method, which has functioned well for centuries.

As has been said before, the problem is that the scientific method is right eventually. It can and often does get stuck for decades at a time, if someone with, shall we say, durable beliefs gets tenure, amasses political power and shoves their rivals out of a field. The amyloid hypothesis is just the most recent example.

Modern metascience practices (preregistration, blinding, banning "garden of forking paths" subgroup analysis, demanding high p factors and larger n) don't replace the scientific method, they're supposed to speed it up! But, by definition, these are all political issues, so they attract political arguments.

Re:No evidence for nudging after adjusting for publication bias by unityByFreedom on Monday July 25, 2022 @10:05PM (#32233420)
> It can and often does get stuck for decades at a time,

I think this is just more data. If we're all wrong for a longer time, then the impact will be more clear.

I agree that modern additions like preregistration are helpful. I only wanted to remark that it is not a prerequisite for science.

> by definition, these are all political issues, so they attract political arguments.

People are good at gaming systems. We are naturals at recognizing patterns and will adjust our behavior to meet our goals. In that sense, social science may be targeting a moving object, almost like the difference between observing and not observing the atoms in a double slit test.

As hard as it may be in social science, the process of hypothesizing, observing and forming conclusions is still science. For some, it appears that is not science because a definite conclusion never arrives.

Which viewpoint is correct? I think it's up to you to decide. And, when you don't grant people that choice, you get an anti-science response, because people naturally reject being told what to think. Science, for me, is about asking questions, not necessarily arriving at a definitive result.

Re:No evidence for nudging after adjusting for publication bias by planck01 on Tuesday July 26, 2022 @01:47AM (#32234729)
> however that overlooks how difficult it is to collect such data.

It is very difficult in physics as well. Do you know how hard it is and how much effort is involved in building the LHC? Or Ligo? Or the JWST? Or ITER? They cost billions of dollars, thousands of scientists and decades to plan and make before you even get science data. Science is hard! You need to put the work and effort in, because otherwise you can't say anything about the nature of things.

Re:No evidence for nudging after adjusting for publication bias by TameAntelope on Sunday July 24, 2022 @09:16PM (#32219544)
> I don't think it matters if this criticism is right or not.

Okay, you and I care about very different things.

Re:No evidence for nudging after adjusting for publication bias by harry8 on Sunday July 24, 2022 @09:23PM (#32219574)
>> I don't think it matters if this criticism is right or not.

> Okay, you and I care about very different things.

Clearly it is being suggested that it doesn't matter with respect to the social studies departments being in dire need of drastic reform. If you don't care about that either direction why are you commenting on this thread? You do actually care one way and you're "point scoring" to further the argument? Something else? I'm misreading something that I think I'm reading clearly?

Re:No evidence for nudging after adjusting for publication bias by TameAntelope on Sunday July 24, 2022 @10:04PM (#32219833)
Hm? I just think it matters if the letter (the original submission) is correct or not.
Re:No evidence for nudging after adjusting for publication bias by harry8 on Monday July 25, 2022 @09:36AM (#32224064)
So are social studies departments and funding in dire need of reform in your opinion? How does the "correctness" in your view of the original submission affect that need or non-need?
Re:No evidence for nudging after adjusting for publication bias by TameAntelope on Monday July 25, 2022 @11:18AM (#32225579)
Not sure I know enough to have an opinion about that; what I've been commenting about is the "nudge is dead" attitude that I don't think is warranted.
Re:No evidence for nudging after adjusting for publication bias by harry8 on Monday July 25, 2022 @09:34PM (#32233255)
and yet here you are responding in a thread with the conclusion clearly stated:

>means it’s past time to reform a large chunk of academia.

And you're taking exception to it, but now just claiming you're actually not, that these one sentence "you're wrong" responses are really something much more modest.

Out of interest do you work in the field? Have ties as a graduate to one of the departments? Or are you completely disinterested when assessing the research?

If I have an interest it's that quality research is performed that advances human knowledge with some kind of efficiency of the resources spend. ie fund something that is being done properly and well to some effect over something that has been shown to be run by those who seem to be utterly incompetent or fraudulent shysters or something else that engenders zero confidence.

https://www.google.com/search?q=cass+sunstein+site%3Astatmod...

Re:No evidence for nudging after adjusting for publication bias by unityByFreedom on Monday July 25, 2022 @11:52PM (#32234054)
> past time to reform a large chunk of academia

>> And you're taking exception to it

TameAntelope was addressing premises to that conclusionary comment.

If the premises do not hold, then TameAntelope does not need to address that conclusion.

Re:No evidence for nudging after adjusting for publication bias by harry8 on Wednesday July 27, 2022 @06:48AM (#32248890)
>If the premises do not hold

But they do tho'

/tips hat to TameAntelope's style

Re:No evidence for nudging after adjusting for publication bias by unityByFreedom on Wednesday July 27, 2022 @10:34PM (#32259879)
TameAntelope's comment did have substance. Give it another read,

https://news.ycombinator.com/item?id=32218408

Re:No evidence for nudging after adjusting for publication bias by harry8 on Thursday August 04, 2022 @02:43AM (#32339912)
one of them did, kind of, sure. Then subsequently no substance or support. Give the thread another read.
Re:No evidence for nudging after adjusting for publication bias by TameAntelope on Tuesday July 26, 2022 @12:21AM (#32234211)
I think you’re on the wrong comment chain, friend…
Re:No evidence for nudging after adjusting for publication bias by harry8 on Wednesday July 27, 2022 @06:54AM (#32248926)
Hey that's just what I said to you! Great minds etc.

Hitch your wagon to Cass Sustein et. al. by all means... Famously successful and successfully famous. What else do you need to know? ESP might be a thing too, nobody has proved it isn't. But we have shut down the research departments at universities involved in that BS...

https://www.google.com/search?q=cass+sunstein+site%3Astatmod...