A machine learning approach to identifying thought markers of suicidal subjects

Re:A machine learning approach to identifying thought marker... by rocqua on Tuesday November 08, 2016 @03:51AM (#12898652)

That 85% 'accuracy' is ambiguous.

I couldn't get into the article, but if that number includes false positives, that's not really practical. Suicide is really rare, so it could just be that it picks a 15% of the population that includes al suicidal people. That means the vast majority of those positives are false positives.

Basically, its the precision and recall that matters.

[ Parent | on HN ]

Re:A machine learning approach to identifying thought marker... by denzil_correa on Tuesday November 08, 2016 @06:58AM (#12899520)

I'm not even sure why people report ONLY accuracy as an evaluation metric. It means nothing - appropriately weighted F scores, precision, recall give more insight into the system.

[ Parent | on HN ]

Re:A machine learning approach to identifying thought marker... by dasboth on Tuesday November 08, 2016 @08:18AM (#12899947)

Completely agree. Although if your binary classes are perfectly balanced then accuracy is a valid metric, but that's rarely the case.

I guess the problem is that an ROC curve wouldn't create flashy headlines.

[ Parent | on HN ]

Re:A machine learning approach to identifying thought marker... by denzil_correa on Tuesday November 08, 2016 @09:31AM (#12900477)

> I guess the problem is that an ROC curve wouldn't create flashy headlines.

You can, you just need to put in a little bit of work. We don't need to specify the exact accuracy in the title.

[ Parent | on HN ]

Re:A machine learning approach to identifying thought marker... by rawnlq on Tuesday November 08, 2016 @06:04AM (#12899249)

I think machine learning people use a combination (specifically the harmonic mean) of both when evaluating:

https://en.wikipedia.org/wiki/F1_score

[ Parent | on HN ]

Re:A machine learning approach to identifying thought marker... by dx034 on Tuesday November 08, 2016 @06:27AM (#12899356)

They should use it, but articles often drop that (or similar) numbers. It's easy to get headlines if you report 85% accuracy as it sounds good and is easy to understand. Reporting the proper numbers is harder to understand for people who are not working in the field and you'd quickly discover that the model doesn't actually work.

[ Parent | on HN ]

Re:A machine learning approach to identifying thought marker... by nommm-nommm on Tuesday November 08, 2016 @09:16AM (#12900347)

I wouldn't call suicide really rare, it's the 10th most common cause of death in the US

http://www.cdc.gov/nchs/fastats/leading-causes-of-death.htm

[ Parent | on HN ]

Re:A machine learning approach to identifying thought marker... by hga on Tuesday November 08, 2016 @09:51AM (#12900648)

Although nowadays you have to view these statistics carefully, for X, Y, and Z causes of death inevitably rise as I, J, and K get knocked down by modern medicine, all we've done to reduce many types of accidental death, from 5 gallon buckets to cars, etc.

But, yeah, in this case I agree it's a big problem in the US.

[ Parent | on HN ]