News for hackers. Stuff that matters.
Re:Programming Languages for Machine Learning by djacobs on Tuesday November 15, 2011 @10:08PM (#3241550)
We're using Matlab/Octave in the Stanford ML course, and it's certainly elegant but not necessarily comfortable for functional programmers. I'm reimplementing our assignments with Clojure/Incanter and am finding my code just as performant and even cleaner. In my opinion, the ML domain maps to Lisp code almost as well as AI as a whole.
Re:Programming Languages for Machine Learning by ericlavigne on Wednesday November 16, 2011 @12:26AM (#3241948)
The instructor emphasized the performance benefit of Octave's optimized matrix operations. Did you use Colt's matrix operations, or were you able to match the performance of vectorized Octave code with non-vectorized Clojure code?
Re:Programming Languages for Machine Learning by aheilbut on Wednesday November 16, 2011 @02:07AM (#3242170)
I think the real magic comes from understanding the algorithms as matrix operations, and then implementing them by more or less just writing down the algebra.
Re:Programming Languages for Machine Learning by billswift on Wednesday November 16, 2011 @04:23AM (#3242458)
Can you give a source for learning about "algorithms as matrix operations"? I tried searching on that phrase in Google Scholar but didn't get anything useful.
Re:Programming Languages for Machine Learning by oscilloscope on Wednesday November 16, 2011 @05:43AM (#3242604)
If you can express the code using the functions and operators of linear algebra, you can simply write the math and get fast, elegant code. May even run on a GPU.

Fortran is frequently used for physics simulation. We used Mathematica, Maple and Matlab even in non-computing classes.

See http://www.netlib.org/lapack/

Also http://en.wikipedia.org/wiki/Array_programming

Re:Programming Languages for Machine Learning by billswift on Wednesday November 16, 2011 @03:06PM (#3244833)
Your first phrase captures what I assumed he meant, I was wondering if there is anything available that could help me recognize algorithms that could or should be expressed that way, and how to do it. Your Wikipedia link was helpful.

I am working on a calculus refresher right now (I really need it) and am planning on working through an elementary linear algebra text once that is done, which is why I particularly noticed the comment. I have found that having an idea of applications helps retention, which is one reason I am trying to track down something more concrete.

Re:Programming Languages for Machine Learning by aheilbut on Saturday November 19, 2011 @02:21AM (#3254840)
Most of the textbooks on machine learning present things in that way (it's really the only way). For example, check out Elements of Statistical Learning (http://www-stat.stanford.edu/~tibs/ElemStatLearn/).
Re:Programming Languages for Machine Learning by jaylevitt on Wednesday November 16, 2011 @08:16AM (#3242969)
If, like me, you thought "linear algebra" meant "stuff without exponents", you should start here:

http://www-math.mit.edu/~gs/papers/starting2matrices.pdf

It's the most accessible starter I've found to date. There is also a linear algebra group-learning thread somewhere on HN.

Re:Programming Languages for Machine Learning by rincewind on Wednesday November 16, 2011 @01:50AM (#3242140)
I also implemented some of the exercises in clojure, and my naive implementation without colt or incanter ran orders of magnitude slower than my octave code.

Then again i did not code it with performance in mind. I overloaded algo.generic for matrix operations, so the fixnum math was probably not inlined.

Re:Programming Languages for Machine Learning by djacobs on Wednesday November 16, 2011 @02:38AM (#3242231)
I used Colt. I'd be interested to see if Clojure parallelization and under-the-hood optimizations could match vectorization performance. Note, I haven't run official benchmarks yet. My comparisons have been more like "back propagation and gradient checking took about the same length of time in both".
Re:Programming Languages for Machine Learning by spariev on Wednesday November 16, 2011 @10:48AM (#3243518)
I believe Incanter uses Colt/Parallel Colt under the hood
Re:Programming Languages for Machine Learning by option1138 on Wednesday November 16, 2011 @12:03AM (#3241882)
I'm in the course as well and don't really understand what the fuss about Octave is. The help system is abysmal and third-party library support seems far behind other popular platforms. It's also quite unstable on Windows.

I've been playing around with R for the past two weeks and have been more or less happy (with the exception of the memory and speed limitations in the GNU implementation).

Nevertheless, this will be a really exciting field for the next decade or so. Amazing possibilities right now!

Re:Programming Languages for Machine Learning by tikhonj on Tuesday November 15, 2011 @10:17PM (#3241571)
Choosing a language for a class has different constraints than choosing one for research or production systems. I imagine performance is less important while ease of learning the language is probably worth much more (probably means not using Haskell for a class :().

I think the ML class at my university--it's in "beta" right now--is using Python following similar logic.

Re:Programming Languages for Machine Learning by tensor on Tuesday November 15, 2011 @11:21PM (#3241757)
The place where performance really matters is generally in the core learning algorithms. Since these are all written in C or Java, you can very easily get away with programming the rest of the code in a higher level language. This is why Python is a feasible choice for machine learning at all.

That said, Clojure is certainly a lot faster than Python for native code and the performance gap with Java is continuing to drop. It's also worth pointing out that you should not ignore C or C++ libraries just because your are running on the JVM. It's not very hard to interface to a core C library with JNI, although be warned, there is a slight trick to doing this via clojure rather than Java.

Re:Programming Languages for Machine Learning by nosignal on Wednesday November 16, 2011 @12:37AM (#3241986)
I'm doing the course as well and have been wanting to become more familiar with Clojure. I'd be very interested to see your implementations of the algorithms in a functional style. After the course is finished, I hope you consider putting up a blog or repo with the solutions.
Re:Programming Languages for Machine Learning by djacobs on Wednesday November 16, 2011 @02:41AM (#3242237)
I'll be happy to do that. If you mail me your contact info, I'll ping you when the class is over.