News for hackers. Stuff that matters.
Re:Introducing HipHip (Array): Fast and flexible numerical c... by peatmoss on Wednesday July 10, 2013 @02:12PM (#6021736)
Looks awesome! One data issue I've seen go relatively unaddressed in the Clojure community is the serialization of big matrices and arrays.

There's a start on a clojure hdf5 (hdf5 is a container format common in scientific circles) implementation, but it's a long ways from done. https://github.com/clojure-numerics/clj-hdf5 I'm not the author, but I am the negligent steward.

I'd love it if someone smarter / better at Clojure than me was interested in helping to think about useful, idiomatic high-level abstractions on top this high-performance data store.

PyTables does a great job of making gobs of hdf5 data easy to work with for analysts--I'm just too novice at Clojure/FP to know what is a reasonable analogue for Clojure.

Re:Introducing HipHip (Array): Fast and flexible numerical c... by prospero on Wednesday July 10, 2013 @02:56PM (#6021987)
Without knowing anything about hdf5 specifically, Vertigo [1] will let you treat a memory-mapped file (or a piece of one) as a normal Clojure data structure, as long as the element types are fixed-layout.

[1] https://github.com/ztellman/vertigo

Re:Introducing HipHip (Array): Fast and flexible numerical c... by Mikera on Thursday July 11, 2013 @03:19PM (#6028482)
I've been thinking about adding some IO features to core.matrix. Haven't got round to it yet, but linking this with hdf5 format could be very useful.