News for hackers. Stuff that matters.
Re:Accelerating Python Libraries with Numba by onalark on Thursday May 09, 2013 @02:23PM (#5682102)
I'm always interested in working with practical examples. Do you have some sample code I could look at?
Re:Accelerating Python Libraries with Numba by tlarkworthy on Friday May 10, 2013 @04:59AM (#5685270)
EDIT: Formatting is a bit whacked. I have two consecutive frames of a sonar image (same size) as the input, you will have to exchange those parts but I ran it fine. There are more parameters in the "DEFAULT_PARAMS" dict as I copied this from a much larger program. I work in greyscale so that might actually be a problem for genralization.

EDIT2: deleted as the src has been truncated, http://pastebin.com/dPMsRF78

Re:Accelerating Python Libraries with Numba by onalark on Monday May 13, 2013 @01:08PM (#5699884)
Hi tlarkworthy, thanks for sharing your code with me!

- It's really hard to make valid comparisons against data I don't have access to. Do you have any open data sets to try this comparison against?

- It looks like the meat of the work is being done here by scikit-learn. As was mentioned earlier, Numba at this stage is mostly useful for improving kernel performance, not large library routines.

I'm planning on taking a deeper look into some of the scikit-learn kernels in the future. Keep an eye open for a blog post from Continuum on this.

Re:Accelerating Python Libraries with Numba by tlarkworthy on Tuesday May 14, 2013 @06:19AM (#5703935)
OK these are the images, feel free to replicate them distribute them, whatever, there is no licensing.:- http://img716.imageshack.us/img716/5808/frame0108.jpg and http://img254.imageshack.us/img254/9562/frame0109.jpg

Yeah I use openCV and scikit learn to do the heavy lifting indeed. But then if I used Numba surely that is what you are advocating too?? I tried unrolling the inner kernel a few different ways before settling on the code you have before you. It doesn't compute the feature vector exactly how I would want, BUT ITS really FAST, which for sonar analysis to run on AUV in real time that's essential. I will fit my math to the library to meet my CPU budgets. Anyway I hope you can use this as a benchmark or something even if its implemented in a totally different way than you might do.

feel free to email me tom dot larkworthy <at> gmail

Tom

Re:Accelerating Python Libraries with Numba by tlarkworthy on Tuesday May 14, 2013 @10:46AM (#5705177)
oh I should also add that the clustering (kmeans) is very fast. The analogous part to the inner loop is "feature_vectors". Which has 2 main cases: case 1, in the case nothing has been computed it calculates the corner response images at all the different spatial scales (big operations). case 2, feature_vector just selects data from the corner images for the pixels demanded.

Now my algorithm is sparse so its normally jsut selecting a subset of pixels, although as it calculated the spatial responses of the whole image it doesn't really make any odds to my runtime.

Re:Accelerating Python Libraries with Numba by onalark on Tuesday May 14, 2013 @01:51PM (#5706646)
Thanks. I'll take a look.
Re:Accelerating Python Libraries with Numba by onalark on Monday May 13, 2013 @10:51AM (#5698967)
Sorry, I missed this reply earlier, I'll take a look at it today.