News for hackers. Stuff that matters.
Re:Saddle: Scala Data Library by aklein on Tuesday April 02, 2013 @02:49PM (#5481329)
Thanks! I will do some follow-up posts on performance, but know that it has been a MAJOR design consideration.

Consider the following in Saddle:

  val s1 = Series(vec.rand(10000), Index(Vec(array.randIntPos(10000)) % 100))

  val s2 = Series(vec.rand(10000), Index(Vec(array.randIntPos(10000)) % 100))

  clock { s1.join(s2, how=index.OuterJoin) }
This clocks in at 19ms on my machine after Hotspot kicks in.

The equivalent pandas:

  In [10]: ix1 = np.random.random_integers(0, 100, 10000)

  In [11]: ix2 = np.random.random_integers(0, 100, 10000)

  In [12]: df1 = DataFrame({'x' : np.random.rand(10000)}, ix1)

  In [13]: df2 = DataFrame({'y' : np.random.rand(10000)}, ix2)

  In [14]: %timeit df1.join(df2, how='outer')
10 loops, best of 3: 37.7 ms per loop
Re:Saddle: Scala Data Library by aklein on Tuesday April 02, 2013 @02:57PM (#5481380)
PS Regarding EJML, after extensive research, I found it hands down the fastest pure-java implementation for doing linear algebra.

While it's maybe 2x-4x slower than JNI wrapped ATLAS or MKL, for the cases I deal with, it just doesn't matter vs ease of use.

That said, it's LGPL, so I made it easy to swap out for other matrix libraries if you need.