Re:Metaprogramming Python For Big Data
by vtuulos on Thursday December 12, 2013 @04:46AM
(#6893653)
Yes, definitely. We might be able to open-source some parts of the system eventually as well.
[ Parent
| on HN ]
Excellent. For sparse storage, do you use Scipy Sparse, or you have your own custom built solution?
I know continuum was working on a version of memmapped numpy (blaze, iirc) arrays which looked really interesting.
I started with Scipy Sparse. I ditched it for a custom solution so I could support variable-length integers, run-length encoding etc.
Good work Ville, let me know when you post those slides/writeup. :-)