What do Data Scientists use to train models fast?

Posted by moridin007 on Sunday August 02, 2015 @09:31AM
from the self-post dept.

i'm training a machine learning model using SVM in python and it took aages for it to happen on my local machine. (with 10% of the data that i have) i'm getting a 80-90% correct prediction score on the same subjects data so now want to add in the rest of the data. (11 more subjects)

i thought of offloading it to my ec2 instance but i'm on a budget so cant just take a 30CPU instance.. on top of everything the code just uses 1 CPU at 100% always so i'm not sure about how effective it would be.

what do you guys use to train these models?

( 7 comments | 6 points | on HN )

Re:What do Data Scientists use to train models fast? by syllogism on Sunday August 02, 2015 @07:14PM (#9992893)

Speed comes from two things: implementation and algorithm. Algorithmically, the way to learn quickly is to use some sort of stochastic gradient method, i.e. learn from examples one-by-one, as opposed to as a batch.

As far as implementation goes, you need dense arrays. A native Python implementation will usually be lists of Python objects, which is very slow.

If you just need an SVM implementation, libsvm is pretty good. I'm assuming you need a non-linear kernel. If you're using a linear kernel then there's not really a difference between SVM and MaxEnt (well, there is but not much).

If your data is very sparse then there aren't many general-purpose implementations that are any good. The scipy.sparse module has some key stuff implemented in pure Python, and doesn't interoperate properly with the rest of the PyData ecosystem. I had to implement my own sparse data structures, in Cython.

[ Parent | on HN ]

Re:What do Data Scientists use to train models fast? by facorreia on Sunday August 02, 2015 @09:51AM (#9990718)

One approach is to convert the code to use parallelism. For an example of how to do it in Python using joblib see this article: http://blog.dominodatalab.com/simple-parallelization/

Even if you can't afford a 32-core instance, you might get to use 4 cores in your laptop.

[ Parent | on HN ]

Re:What do Data Scientists use to train models fast? by syllogism on Sunday August 02, 2015 @07:15PM (#9992897)

The problem's pretty obviously the Python implementation...Throwing more cores at it isn't really going to help.

Is it even easy to parallelise SVM training?

[ Parent | on HN ]

Re:What do Data Scientists use to train models fast? by moridin007 on Monday August 03, 2015 @01:07AM (#9993925)

so the solution would be move from python to go? like python isn't that good for ML algorithms?

[ Parent | on HN ]

Re:What do Data Scientists use to train models fast? by syllogism on Tuesday August 04, 2015 @12:46PM (#10004531)

Python's fine if the heavy lifting is in a C extension. I use Cython for this; others prefer just numpy, maybe something like numba. But you can't just have a Python list of floats.

Java, C++, Scala and Julia are all popular choices. Go is probably fine too, although I know less about it.

[ Parent | on HN ]

Re:What do Data Scientists use to train models fast? by Lofkin on Friday August 07, 2015 @11:31AM (#10023026)

Actually list support in numba is WIP

[ Parent | on HN ]

Re:What do Data Scientists use to train models fast? by rajacombinator on Monday August 03, 2015 @03:15AM (#9994236)

How much data and how long are you talking about? If it fits in memory, then the slowness is likely due to other coding errors causing a bottleneck, not the SVM training. (Unless you wrote that as well.)

[ Parent | on HN ]