News for hackers. Stuff that matters.

A comment on A Quick Look at Support Vector Machines. Parent comment

Re:A Quick Look at Support Vector Machines by inlineint on Thursday March 23, 2017 @11:05AM (#13940339)
On the other hand SVM doesn't scale as well as neural networks do because it has computational complexity between O(n^2) and O(n^3) [1] where n is the number of samples in the training set. So if you plan to add more data later you may eventually encounter scaling problems with SVM.

[1] http://scikit-learn.org/stable/modules/svm.html#complexity

[ Parent | on HN ]
Re:A Quick Look at Support Vector Machines by samuell on Thursday March 23, 2017 @12:10PM (#13941062)
We found great success with the LIBLINEAR SVM implementation [1] though: Extremely good performance, to the point that it affects scalability too, with predictive performance acceptably close to libSVM with the RBF kernel, for a large cheminformatics dataset:

Paper (open acccess): http://dx.doi.org/10.1186/s13321-016-0151-5

As can be seen in fig 5 [2] in the paper, a dataset size that took ~1 week with libSVM (actually, the parallel piSVM implementation) on 64 cores, took less than a minute with LIBLINEAR, which runs on just one core.

[1] https://www.csie.ntu.edu.tw/~cjlin/liblinear

[2] http://jcheminf.springeropen.com/articles/10.1186/s13321-016...

[ Parent | on HN ]
Re:A Quick Look at Support Vector Machines by Fede_V on Thursday March 23, 2017 @04:35PM (#13944001)
With an RBF Kernel, you are stuck solving the dual problem which has O(n^2 * m) complexity. With the linear kernel, you can solve the primal scales with O(n*m) complexity.
[ Parent | on HN ]
Re:A Quick Look at Support Vector Machines by syntaxing on Thursday March 23, 2017 @11:24AM (#13940541)
Good to know, I did not know that! I kind of wish scikit had some sort of CUDA capabilities to speed things up.
[ Parent | on HN ]
Re:A Quick Look at Support Vector Machines by nl on Thursday March 23, 2017 @06:10PM (#13944811)
Scikit uses numpy which uses BLAS which can be implemented with nvBLAS[1]. I don't know what it takes to get this setup and how much performance boost it gives for SVMs though.

[1] https://developer.nvidia.com/cublas

[ Parent | on HN ]