Re:BayesDB: Data science is a communication problem
by mcbrown on Thursday March 31, 2016 @08:47PM
(#11401776)
BayesDB and BQL are great ideas in theory. But the problem with building any "general" analysis tool to abstract away underlying complexity is that it must either be (a) a least common denominator solution, or (b) the proverbial hammer to which everything will look like a nail.
The reason why there are lots of different statistical methods is that different problems and different samples from different populations call for different approaches. And frequently the reason why a certain approach is invalid in a particular use case is subtle, difficult to explain, and easy to miss. Abstracting that complexity away from the would-be "scientist" is an invitation for them to develop unreasonable confidence in their results "because the model said so." We have more than enough of that attitude already, thank you very much.
The reason why there are lots of different statistical methods is that different problems and different samples from different populations call for different approaches. And frequently the reason why a certain approach is invalid in a particular use case is subtle, difficult to explain, and easy to miss. Abstracting that complexity away from the would-be "scientist" is an invitation for them to develop unreasonable confidence in their results "because the model said so." We have more than enough of that attitude already, thank you very much.