News for hackers. Stuff that matters.
Re:Ketamine lifts rodents' mood only if administered by male... by arkades on Tuesday November 21, 2017 @11:37PM (#15754279)
Hardly new. Feynman was criticizing psych research for this sort of thing - inadequate controls - half a century ago in his talk on Cargo Cult Science. http://calteches.library.caltech.edu/51/2/CargoCult.htm
Re:Ketamine lifts rodents' mood only if administered by male... by raverbashing on Wednesday November 22, 2017 @06:05AM (#15755921)
And what happens when you have more variables than you can control?

It's naive to think all of them can be controlled, even in simple animals as mice

Re:Ketamine lifts rodents' mood only if administered by male... by arkades on Wednesday November 22, 2017 @11:46AM (#15758219)
The problem isn’t having variables you can’t control. The problem is not recognizing these variables, or recognizing them and being too lazy to control for them, and then drawing unwarranted, overly broad hypotheses.

There’s a reason psych has a bigger reproducibility problem than any other field.

Re:Ketamine lifts rodents' mood only if administered by male... by yjftsjthsd-h on Wednesday November 22, 2017 @07:00AM (#15756126)
> when you have more variables than you can control

Worse: isn't this always true? Specific breed of mice, humans working with them, temperate, trace differences in food, light patterns, sounds (including sounds beyond human range)...

Re:Ketamine lifts rodents' mood only if administered by male... by SomeStupidPoint on Wednesday November 22, 2017 @06:32AM (#15756010)
Then it's outright fraud to present your results as reliable when there's radically uncontrolled variables that dominate the results.
Re:Ketamine lifts rodents' mood only if administered by male... by Cthulhu_ on Wednesday November 22, 2017 @06:58AM (#15756119)
Just make sure to note the variables if you can list them. The main problem is media outlets taking the results as fact.
Re:Ketamine lifts rodents' mood only if administered by male... by nonbel on Wednesday November 22, 2017 @08:00AM (#15756394)
This is also naive. In that case you won't be able to publish your results in prestigious journals and your career will die out.

That said, you do learn not to act 100% confident. You need to say "This suggests that" instead of "This proves that", etc.

Anyway, the solution to reliable data is simple: Have multiple competing groups try to repeat the work as closely as possible.

Now that you have something to hang your hat on, you come up with some theory/model to explain the phenomenon. Even better, come up with multiple competing such models. From these you deduce some precise prediction that is checked against future data.

All of this was well known a few generations ago...

Re:Ketamine lifts rodents' mood only if administered by male... by SomeStupidPoint on Wednesday November 22, 2017 @10:59AM (#15757760)
> This is also naive. In that case you won't be able to publish your results in prestigious journals and your career will die out.

Charlatan fortunetellers aren't owed careers as scientists -- if they can't produce reliable, predictive models because they utterly fail to control experiments,then they should have their careers destroyed.

We've seen enough damage from corporate leaders and politicians relying on guides to reading chicken entrails masquerading as science.

It's time to stop the fraud.

Re:Ketamine lifts rodents' mood only if administered by male... by nonbel on Wednesday November 22, 2017 @11:24AM (#15758017)
Sure, just be aware of what you are saying. Over 99% of what gets published is like this. Generations have wasted their careers producing misinformation. It all needs to be redone since these practices began (in some areas of research this is at t=0). Etc.
Re:Ketamine lifts rodents' mood only if administered by male... by majewsky on Wednesday November 22, 2017 @11:11AM (#15757875)
Consider the consequences of what you're demanding: You'd basically outlaw any research on complex systems. Everything in biology, sociology, archeology, climate science, etc. Is that a reasonable response to the problem at hand?
Re:Ketamine lifts rodents' mood only if administered by male... by nonbel on Wednesday November 22, 2017 @11:26AM (#15758043)
Why would it be outlawed?
Re:Ketamine lifts rodents' mood only if administered by male... by SomeStupidPoint on Wednesday November 22, 2017 @11:29AM (#15758069)
No, I would bifurcate research into two classes --

Let's call it 'science' and 'hunches'. (I'm sure marketing people would think of a better name.) There's nothing stopping you from publishing hunches; there's nothing stopping the same journals from publishing science and hunches. It would just be the case that we'd treat publishing a hunch as science as fraud, because you're outright lying about how sure you are.

It is the case that several fields would get (almost entirely) moved from science to hunches, but that would be good! The label will match what's inside.

If we have this split in the product lines, if we have these two tiers (models where we're really sure and complex systems where we only have a working hunch), then we should just own it and put that on the tin.

All I'm asking is people stop labeling composite wood as lumber -- and understandably, a lot of people selling composites act like that would make the sky fall.

Hardly, there's a real demand for both products and both serve a real need. Let's just be honest in our marketing and stop conflating the two.

Re:Ketamine lifts rodents' mood only if administered by male... by sideshowb on Wednesday November 22, 2017 @06:57AM (#15756113)
It would only be fraud if done with deliberate intent to deceive.
Re:Ketamine lifts rodents' mood only if administered by male... by mayank on Wednesday November 22, 2017 @01:06AM (#15754657)
Inadequate controls = attempting to measure fine effects (gender normalized animal behavior) using crude methods (rat study).
Re:Ketamine lifts rodents' mood only if administered by male... by colehasson on Wednesday November 22, 2017 @02:05AM (#15754906)
Accuracy or precision. Pick one?
Re:Ketamine lifts rodents' mood only if administered by male... by marcosdumay on Wednesday November 22, 2017 @08:04AM (#15756423)
Well, why should anybody not pick accuracy every time?
Re:Ketamine lifts rodents' mood only if administered by male... by kbutler on Wednesday November 22, 2017 @08:34AM (#15756620)
Where is the closest gas station? "Within the Solar System".
Re:Ketamine lifts rodents' mood only if administered by male... by nerdponx on Wednesday November 22, 2017 @08:16AM (#15756501)
Because, in the extreme, accuracy without precision doesn't tell you a damn thing.
Re:Ketamine lifts rodents' mood only if administered by male... by marcosdumay on Wednesday November 22, 2017 @08:38AM (#15756653)
Just like precision without accuracy, but if you get accuracy you know you have a problem, but with only precision you are hopelessly misled.

If it was a strict trade-off, I would agree that the answer was somewhere on the middle. But it isn't. Most of the things that give you precision will also give you accuracy. The few things that let you exchange one for the other look too much like cheating, and seem to always have much greater impacts on accuracy than on precision.

Re:Ketamine lifts rodents' mood only if administered by male... by kbutler on Wednesday November 22, 2017 @12:03PM (#15758348)
> Most of the things that give you precision will also give you accuracy.

This is absolutely false - digital clocks are a common example. Precise to the second (or millisecond, or more!) but only as accurate as their setting.

Calculations also give precision without accuracy, like my package of tortilla chips - 13oz (368.5g) They took a measurement or specification precise to the ounce, multiplied by 28.3495, and got 368.5.

Accuracy and precision are independent variables, and the utility of either is usually limited to the order of magnitude of the other. With either "accurate to the second with a precision of milliseconds", or "accurate to the millisecond with precision to the second", you should only report a measurement to the second (or possibly a more precise measurement with an error range).

Re:Ketamine lifts rodents' mood only if administered by male... by felipemnoa on Wednesday November 22, 2017 @09:20AM (#15756931)
>>Accuracy or precision

Do they have different meaning to you? They are the same thing unless there is some scientific meaning that makes them different of which I'm not aware. i.e. Theory in Science means a different thing than what it means to a layman person.

Re:Ketamine lifts rodents' mood only if administered by male... by jodoherty on Wednesday November 22, 2017 @10:00AM (#15757203)
Precision is how fine a measurement is (i.e. being able to measure temperature to thousandths of degrees versus tenths), whereas accuracy is a measure of how closely your results much the objective truth. So you may have a very precise thermometer that measures to thousandths of degrees but it may not be accurate because it's calibrated such that it always reads exactly two degrees higher than it should. Meanwhile a thermometer that can only measure to a tenth of a degree is more accurate and less precise than the one just mentioned if the shown figures reflect the true temperature.

In common usage, these words mean the same thing, but scientists often give words specific meanings in order to make their research and ideas less ambiguous.

Definitions are even more important in non-scientific fields such as philosophy or math where meaningful reasoning of abstract structures and ideas would be next to impossible without giving them concrete definitions and stating your assumptions.

Remember that definitions are arbitrary, so to understand an author's argument or idea, you must seek out the author's definitions.

Hopefully that clarifies some of the discussion going on in the comments here.

Re:Ketamine lifts rodents' mood only if administered by male... by felipemnoa on Wednesday November 22, 2017 @01:01PM (#15758896)
I get it, these are specific definitions within a given field. I looked at the definition of both precision [1] and accurate [2] and each would use the other as a synonym, which made it a bit confusing as to what he was talking about.

[1] pre·ci·sion: noun the quality, condition, or fact of being exact and accurate.

[2]ac·cu·rate (of information, measurements, statistics, etc.) correct in all details; exact. "accurate information about the illness is essential" synonyms: correct, precise, exact, right, error-free, perfect; More

Re:Ketamine lifts rodents' mood only if administered by male... by jodoherty on Wednesday November 22, 2017 @05:27PM (#15761235)
Standard English dictionaries often don't have the precise definitions used in specific fields, which only leads to more confusion for laymen, but any good introductory Chemistry lab textbook should be able to get you up to speed on experimental science definitions.
Re:Ketamine lifts rodents' mood only if administered by male... by gnajob on Wednesday November 22, 2017 @09:55AM (#15757167)
Re:Ketamine lifts rodents' mood only if administered by male... by kbutler on Wednesday November 22, 2017 @12:15PM (#15758463)
After your watch battery dies, when you change the battery in your watch, it continues to have a 1 second precision.

But it's inaccurate by many hours because you haven't set it yet.

Re:Ketamine lifts rodents' mood only if administered by male... by macintux on Wednesday November 22, 2017 @09:56AM (#15757171)
I admit I'm only skimming the discussion so I may be missing the point of your question, but a common analogy to the difference:

If you're target shooting, and all of your shots go through the same hole: that's precision.

If that hole is several inches away from the bullseye, it's precise but not accurate.