News for hackers. Stuff that matters.
Re:InfluxDB – Open-source distributed time-series, events, a... by pauldix on Tuesday November 05, 2013 @04:14PM (#6678362)
The numpy style interface might be interesting. The problem is that it would probably kill performance to marshal into a Python process. Unless there's some way to embed numpy?

Supporting custom functions is definitely something we want to do.

Re:InfluxDB – Open-source distributed time-series, events, a... by yummyfajitas on Tuesday November 05, 2013 @06:07PM (#6679159)
Why would it kill performance? All you need to do is this:

    double* values = (double*)malloc(sizeof(double)*num_of_values);
(If you are unfamiliar with numpy, it's just a python wrapper around raw blocks of memory.)
Re:InfluxDB – Open-source distributed time-series, events, a... by waterlion on Wednesday November 06, 2013 @05:23AM (#6681937)
You still have to get the bytes out of the server and into the Python process.
Re:InfluxDB – Open-source distributed time-series, events, a... by yummyfajitas on Wednesday November 06, 2013 @06:15AM (#6682090)
Assuming Python is not running in the same process as InfluxDB, amend my code snippet:

    double* data = shmget(key, sizeof(double)*num_data_pts, whatever_flag);
But directly embedding a Python interpreter is pretty easy in C, so I imagine it should also be fairly easy in Go as well.

(Of course, given that my code snippets are C, you can probably deduce I've never written any Go, so take my comments on how easy it is with a grain of salt.)

Re:InfluxDB – Open-source distributed time-series, events, a... by waterlion on Wednesday November 06, 2013 @06:39AM (#6682166)
Go interop with C is very doable (although I've never tried it). Not sure about shared memory WRT garbage collection though.
Re:InfluxDB – Open-source distributed time-series, events, a... by lotyrin on Tuesday November 05, 2013 @06:39PM (#6679348)
A route to this would be providing an FFI for loadable modules that provide custom procedures or functions.

Folks that wanted to could then write Python/Numpy code for these and use Numba+LLVM to compile them.

It could be quite performant and avoid having to marshal data or do IPC, could possibly avoid even copying data in some cases (Numpy/Numba have pretty robust support for structures coming over an FFI, not sure about FFI in Go.)