News for hackers. Stuff that matters.
Re:Introducing Streaming K-Means in Spark MLlib 1.2 by tlarkworthy on Wednesday January 28, 2015 @03:37PM (#8962402)
a general drawback of k-means. You have to select the k. In practice you try a range and see how well they summarize your data (e.g. leave one out cross validation).

You could do that here too, just have a range of k. If only there were a streaming leave-one-out cross validation for k-means to complement this approach ...

(it is possible to do this in a streaming style, see LWPR)

Re:Introducing Streaming K-Means in Spark MLlib 1.2 by michaelmior on Wednesday January 28, 2015 @03:55PM (#8962495)
I understand that's a drawback of k-means. I was just wondering if this was something that Spark solved natively. Looks like the answer is no for the time being. Thanks for the pointer to LWPR :)
Re:Introducing Streaming K-Means in Spark MLlib 1.2 by rxin on Wednesday January 28, 2015 @03:58PM (#8962508)
Maybe using BP-Means: http://arxiv.org/abs/1212.2126

K-Means without preset K.