Re:Can science solve an 80-year-old Indian mystery?
by nerdponx on Tuesday October 04, 2016 @11:43AM
(#12636367)
> The ISV (Inter-Session Variability) algorithm provided a decent comparison score of 0.79 for Kanchan Prabha Devi which was much higher than an image of Zubeida Begum Dhanrajgir. However, the results of the CNN (Convolutional Neural Network) algorithm were not as clear-cut. Devi only managed a score of 0.42 while Zubeida got 0.50.
> So, which algorithm to trust?
This is an excellent example of the danger in using algorithms that produce point estimates, but not variability estimates. Whether you model it or not, any inference or prediction exercise implies a probability model (else you wouldn't need to conduct inference at all). Failing to make that model explicit, whether by nested cross-validation or an analytical technique, leads you down a dangerous road of making arbitrary comparisons And breaking near-ties on the basis of "field knowledge", which can turn out to be a little more than glorified gut instinct.
Re:Can science solve an 80-year-old Indian mystery?
by sandGorgon on Tuesday October 04, 2016 @01:08PM
(#12637114)
Thank you for this!
we are planning to have a consultant come in a build image comparison of selfies with ID cards (for a fintech mobile app). What kind of validation and variation scores should we ask for to explicitly model the probability ?
Re:Can science solve an 80-year-old Indian mystery?
by nerdponx on Tuesday October 04, 2016 @01:46PM
(#12637547)
I don't think there's anything available for CNNs other than resampling techniques like nested cross-validation.
I'm also a fan of sensitivity analysis, where you jitter one of the inputs, or one of the model parameters, and see how robust the output is to local noise.
ISV I'm not explicitly familiar with. But there's no reason you couldn't use one of the general purpose techniques from above.
That said, resampling is very computationally intensive, so I can understand why it's not that popular in conjunction with similarly intensive iterative fitting algorithms like backpropagation and MCMC. I didn't mean my post to be a criticism of the research as much as I wanted to just put a warning out there about an issue that researchers without deep statistical backgrounds tend to overlook.
Re:Can science solve an 80-year-old Indian mystery?
by grif-fin on Tuesday October 04, 2016 @11:00AM
(#12636046)
Not getting into the story just commenting on the title 'Can Science solve...'. It boils my curiosity when sentences start with 'Can science solve X, Y or Z?'. It is referred to as if science is a tool.
I mean is there an alternative to science that we question in that way? -_-
Re:Can science solve an 80-year-old Indian mystery?
by dspillett on Tuesday October 04, 2016 @12:25PM
(#12636682)
> It is referred to as if science is a tool.
Science is a tool, especially when used in such a directed investigative manner.
> I mean is there an alternative to science that we question in that way?
Historical research in this case. Though in a less polarised view of reality both such research and science go hand-in-hand (science questioning the sources, the sources suggesting things for science to test, new sources being found (via archaeology or dredging libraries and other meta-analysis?) perhaps with the help of directions from the science, and so on.
Re:Can science solve an 80-year-old Indian mystery?
by nerdponx on Tuesday October 04, 2016 @11:38AM
(#12636324)
Sure there is! Idle philosophizing, gut instinct, and preconceived notions are all well-established alternatives to scientific inquiry. We note their extensive use in related literature dating back millenia.
Re:Can science solve an 80-year-old Indian mystery?
by chc on Tuesday October 04, 2016 @12:53PM
(#12636948)
In the future we will have new techniques that current-day science does not have, and things that science can't do today, it may be able to do then. The question is whether science can do it now, not whether science is an appropriate avenue of inquiry.
Re:Can science solve an 80-year-old Indian mystery?
by grif-fin on Tuesday October 04, 2016 @01:06PM
(#12637092)
> The question is whether science can do it now, not whether science is an appropriate avenue of inquiry.
I agree but it still does not make the title 'Can science....' a good title to like. A good title/question should be readable at any day and age. This form titles/questions can cause such arguments as we are having, not very productive ones.
> So, which algorithm to trust?
This is an excellent example of the danger in using algorithms that produce point estimates, but not variability estimates. Whether you model it or not, any inference or prediction exercise implies a probability model (else you wouldn't need to conduct inference at all). Failing to make that model explicit, whether by nested cross-validation or an analytical technique, leads you down a dangerous road of making arbitrary comparisons And breaking near-ties on the basis of "field knowledge", which can turn out to be a little more than glorified gut instinct.
we are planning to have a consultant come in a build image comparison of selfies with ID cards (for a fintech mobile app). What kind of validation and variation scores should we ask for to explicitly model the probability ?
I'm also a fan of sensitivity analysis, where you jitter one of the inputs, or one of the model parameters, and see how robust the output is to local noise.
ISV I'm not explicitly familiar with. But there's no reason you couldn't use one of the general purpose techniques from above.
That said, resampling is very computationally intensive, so I can understand why it's not that popular in conjunction with similarly intensive iterative fitting algorithms like backpropagation and MCMC. I didn't mean my post to be a criticism of the research as much as I wanted to just put a warning out there about an issue that researchers without deep statistical backgrounds tend to overlook.
I mean is there an alternative to science that we question in that way? -_-
Science is a tool, especially when used in such a directed investigative manner.
> I mean is there an alternative to science that we question in that way?
Historical research in this case. Though in a less polarised view of reality both such research and science go hand-in-hand (science questioning the sources, the sources suggesting things for science to test, new sources being found (via archaeology or dredging libraries and other meta-analysis?) perhaps with the help of directions from the science, and so on.
Challenging science is absurd as is challenging the reality.
It is challenging the idea that the president can outrun a cheetah, however. So, "Can science solve X?" is not the same as "Is science effective?".
I agree but it still does not make the title 'Can science....' a good title to like. A good title/question should be readable at any day and age. This form titles/questions can cause such arguments as we are having, not very productive ones.
It's just a human effort. That's all.