News for hackers. Stuff that matters.
Re:Goodbye, Data Science by whatever1 on Tuesday November 29, 2022 @10:08AM (#33788462)
See? “Better luck next time”.

Not being mean to you, just showing how typically the goal posts are moved.

To give you an example from physics, if you find just one experiment that goes against your model, you immediately invalidate the model. You don’t just make grand claims that the model in general works.

Re:Goodbye, Data Science by cjglo on Tuesday November 29, 2022 @10:50AM (#33789025)
Could be wrong here, but in physics and most natural sciences, you don’t throw away your model if you have one experiment against it.

Usually isn’t it looking for an experiment that proves it and is repeatable?

If I discover a new element in one experiment, the results are published.

After publication, many labs will try to repeat and its not taken away if one can’t do it. Only if all can’t and it casts doubt on whether I did it in the first place.

Re:Goodbye, Data Science by Jensson on Tuesday November 29, 2022 @06:24PM (#33794827)
Scientific method? Models are disproved, not proved. Do data scientists not know about science? People usually understand how science works here on HN, but not in this thread.

Example of a test that invalidated our old theory of gravity and validated Einsteins claims:

https://en.wikipedia.org/wiki/Eddington_experiment

This is how science is done. But apparently not data science.

Re:Goodbye, Data Science by caddemon on Tuesday November 29, 2022 @08:45PM (#33795883)
Models can also be knowingly/intentionally incomplete, which necessarily introduces noise that you are not controlling for. Meaning you have to use statistics, and there isn't really a concept of prove or disprove in the true sense of those words.

Maybe very far in the future there will be models of human biology that are as robust as classical physics, but right now there is such a large amount that is not understood, it's simply not feasible. A drug could work for one person and not another for reasons beyond the realistic scope of the original development hypothesis. It requires a probabilistic view to make any sort of statement about the efficacy then.

I suppose you could argue these models are just wrong and thus trivially disproven, but I don't think that's a productive framing. I doubt any biologist or doctor would claim they have anywhere near a complete model of how their specialty works. That doesn't mean a particular model isn't useful or isn't the best we currently have to work with.

Plus maybe the third best model will actually turn out to explain a separate puzzle piece in an eventual better model. Mechanistic models in biology aren't always well done in practice, but it's certainly not binary either.

Re:Goodbye, Data Science by md224 on Tuesday November 29, 2022 @11:46AM (#33789874)
> if you find just one experiment that goes against your model, you immediately invalidate the model

Pierre Duhem would like to have a word with you:

https://plato.stanford.edu/entries/scientific-underdetermina...

> Holist underdetermination ensures, Duhem argues, that there cannot be any such thing as a “crucial experiment”: a single experiment whose outcome is predicted differently by two competing theories and which therefore serves to definitively confirm one and refute the other.

Re:Goodbye, Data Science by lagt_t on Tuesday November 29, 2022 @11:55AM (#33790019)
Quantum mechanics says otherwise
Re:Goodbye, Data Science by bart_spoon on Wednesday November 30, 2022 @10:11AM (#33801779)
The goal posts are only moved if they were in the incorrect position in the first place. Statistics isn’t the science of prediction, it’s the science of uncertainty management. There is always uncertainty, and how well you’ve measured uncertainty only can be accurately assessed over a large enough time frame over a large enough number of events.

It’s like when people got upset about Trump winning when 538 only gave him a 30% chance. That one event tells us nothing. But if all predictions 538 says have a 30% chance of occurring happen 30% of the time, then they are spot on. That’s not apparent with a single event though.

The problem is that most managers, companies, and people (including yourself, apparently) are statistically illiterate enough to not understand this, and jump head first into data science initiatives expecting immediate results, which is usually doomed to fail, at which point they blame others and not their poorly formed expectations.

There’s plenty of bad data science out there, but most failed data science initiatives are doomed before anyone every builds a model or analyzes any data.

Re:Goodbye, Data Science by whatever1 on Wednesday November 30, 2022 @12:47PM (#33804498)
Whoa, slow down with your elitism.

Who do you think you are?

Re:Goodbye, Data Science by bart_spoon on Saturday December 03, 2022 @10:07AM (#33844000)
What elitism?