Re:Idea Week. Post ideas you don't plan on implementing.
by fauigerzigerk on Tuesday October 09, 2007 @11:26AM
(#65282)
I'm not sure whether you are referring to the same argument, but I think I remember that Bruce Schneier talks about the lack of correct positives that makes the application of data mining methods to terrorist incidents useless. He makes the point that very few positives for the system to learn from combined with a very large number of variables causes a huge number of false alarms (not sure if I remember all he said correctly).
So it's all about how many confirmed positives you have and whether you have reliable data on them. It works very well for credit card fraud because there is a sufficiently large number of actual fraud cases, you have reliable data about them and false alarms do not cause major disruption.
I agree with your concern about the reliability and completeness of data. I still think it's an interesting idea if there was a way to extract the data from a reliable source instead of working with what people claim to be the case.
So it's all about how many confirmed positives you have and whether you have reliable data on them. It works very well for credit card fraud because there is a sufficiently large number of actual fraud cases, you have reliable data about them and false alarms do not cause major disruption.
I agree with your concern about the reliability and completeness of data. I still think it's an interesting idea if there was a way to extract the data from a reliable source instead of working with what people claim to be the case.