News for hackers. Stuff that matters.
Re:Surges of cosmic radiation from space directly linked to ... by bryan0 on Friday June 16, 2023 @05:45PM (#36364361)
> One thing that troubles me in the paper is that the researchers appear to have gone looking for precursor patterns in an ad hoc way, with no physical theory in mind, just trying different binning techniques and delays until they got a signal.

I believe in particle physics this is known as the “look-elsewhere effect”. Basically if look long and hard enough for a pattern, you will eventual find it if your parameter space is large enough: https://en.m.wikipedia.org/wiki/Look-elsewhere_effect

Re:Surges of cosmic radiation from space directly linked to ... by madrox on Friday June 16, 2023 @06:09PM (#36364600)
When I was taught statistics, this was regularly brought up as a big no-no in science. However, I read a guide to practical statistics that had a gem about predicting the stock market. If we discovered surges in the market correlated to newspaper sales, we wouldn't discard this as look-elsewhere. In fact, we'd follow newspaper sales very closely.

Predicting earthquakes has a big upside for humanity. If there is even a small correlation -- even if we don't yet understand it -- we can benefit from it.

Re:Surges of cosmic radiation from space directly linked to ... by rcxdude on Friday June 16, 2023 @07:26PM (#36365370)
It can be useful, it's just you need to keep monitoring it: if the effect seems to shrink or disappear on future tests, it's probably spurious.
Re:Surges of cosmic radiation from space directly linked to ... by mnky9800n on Friday June 16, 2023 @06:32PM (#36364847)
Most of particle physics is people searching for jobs that give meaning to their life. This often biases them against finding results against meaning.
Re:Surges of cosmic radiation from space directly linked to ... by idlewords on Saturday June 17, 2023 @12:21AM (#36367256)
Not to mention that they are themselves made of the particles they study, which ineradicably biases their outlook.
Re:Surges of cosmic radiation from space directly linked to ... by mnky9800n on Saturday June 17, 2023 @09:20AM (#36369953)
Never be made out particles. That's true.
Re:Surges of cosmic radiation from space directly linked to ... by jaggederest on Friday June 16, 2023 @05:59PM (#36364498)
This is related to the fact that in a higher-dimensional space the hamming distance between two points is compact, compared to 2d/3d intuition. This problem is related to the way that e.g. embedding text into an AI for comparison purposes often produces surprisingly closely related vector distances for relatively unrelated strings.
Re:Surges of cosmic radiation from space directly linked to ... by Ringz on Friday June 16, 2023 @06:04PM (#36364543)
Can we transfer this to the financial markets?
Re:Surges of cosmic radiation from space directly linked to ... by GolDDranks on Friday June 16, 2023 @05:59PM (#36364492)
The obligatory XKCD: https://xkcd.com/882/
Re:Surges of cosmic radiation from space directly linked to ... by breck on Friday June 16, 2023 @06:17PM (#36364700)
In GWAS studies there's a nice visualization for dealing with this called Manhattan Plots. https://en.wikipedia.org/wiki/Manhattan_plot

Basically if you test a lot of hypotheses plot all the p-values and look to see if there is something that is a true outlier or whether it is expected given so many tests.