News for hackers. Stuff that matters.
Re:Outperforming everything with anything: Python? Sure, why... by comicjk on Sunday January 27, 2019 @04:44PM (#19013965)
Learning C is easy, but getting a piece of C integrated with your Python code isn't. It can be done well, but bad software is littered with things that could have been done well. I recently had a project at work of ripping out a bad C implementation of a high-performance section of code and replacing it with faster and shorter piece of numpy, so this is definitely a practical issue.
Re:Outperforming everything with anything: Python? Sure, why... by mlthoughts2018 on Sunday January 27, 2019 @06:07PM (#19014321)
I find the opposite to be true. With numba & Cython, it’s ridiculously simple to target a segment of Python code for compiled specialization, often required for custom pieces of work not directly expressible in NumPy, especially when processing non-ndarray data structures.

I find I am frequently removing sections of inscrutable “vectorized” NumPy code that is inefficient, not because NumPy is inefficient but because conforming the data to a structure amenable to NumPy is too inefficient in the first place, and rewriting in straightforward, simple Cython functions.

Re:Outperforming everything with anything: Python? Sure, why... by carlmr on Monday January 28, 2019 @04:56AM (#19016281)
>I find I am frequently removing sections of inscrutable “vectorized” NumPy code that is inefficient, not because NumPy is inefficient but because conforming the data to a structure amenable to NumPy is too inefficient in the first place

Also the next maintainer will thank you for it.

Re:Outperforming everything with anything: Python? Sure, why... by yvdriess on Monday January 28, 2019 @03:38PM (#19020851)
To generalize, learning C is easy, but doing anything with it is not.

To be clear, this is not meant as a critique on C, on the contrary. Put simply, it is just that making computers do stuff is hard. This is especially true if you want it to do it in a very particular way such as in high-performance code. If you cannot already leverage the work of other experts, you need to know about modern optimizing compilers, language runtimes, modern processor architectures, OS/system, ...)

To continue your Numpy example, that framework entombs the know-how and man-years of work of specialist in a certain area. If you need to do simple dense algebra, you will a hard time beating Numpy/Julia/Eigen/MKL etc. If your problem goes slightly off-track (e.g. sparse problems) where you cannot easily leverage the hard work and expertise of others, you better start rolling up your sleeves and open that main.c

Re:Outperforming everything with anything: Python? Sure, why... by comicjk on Wednesday January 30, 2019 @07:20AM (#19034472)
I used scipy's sparse matrices - that was actually a key part of the project.