One data point to support your claim:
* pure Python: 1.0x (baseline)
* NumPy (numeric python): 120x
* Google-optimized pure C: 2,500x
* optimized Python + Cython + BLAS: 8,700x
This is optimizing the same numeric algorithm (word2vec) using different languages and tricks, from naive pure-Python down to compiled Cython with optimized CPU-specific assembly/Fortran (BLAS) for the hotspots [0].
[0] http://rare-technologies.com/word2vec-in-python-part-two-opt...
Re:“CPUs are optimized for video games”
by flamedoge on Thursday August 11, 2016 @04:20AM
(#12266853)
and I'm sure pure Fortran would be even faster
Re:“CPUs are optimized for video games”
by alfalfasprout on Thursday August 11, 2016 @05:46PM
(#12271815)
Eh, doubtful. Sure, it'll be faster than unoptimized C. But when you observe strict aliasing rules in C a modern compiler will generally produce similar quality assembly for both languages. The reason BLAS is so fast is that every single line of code in a hot loop has been hand-optimized and tons of CPU-specific optimizations exist
Fortran yields some of the fastest code out of the box, yes, but C/C++ can yield equivalent performance by adhering to certain basic rules.
Even with strict aliasing, the only way C/C++ standard effectively lets one type to be cast to another is incurring a copy. Tell me what can be done better for performance in C that can't be done in Fortran or "another static language without pointer indirection".
That's mostly because of Fortran doesn't have the same aliasing trouble as C?
Pointer indirection is expensive tradeoff for flexibility
Re:“CPUs are optimized for video games”
by jeremiep on Thursday August 11, 2016 @10:04AM
(#12268288)
You can get a lot of performance back just by taking branch predictions and cache misses into your designs.
I did just that in C# last years and got at least two orders of magnitude of performance gain from it. Still, I constantly wished I was working in C instead :)
Hah, in my experience that's hopelessly optimistic. The C I write is probably 10-100x slower than it should be (leaving SSE and threading aside entirely), but the Python I write most days is a hundred times slower still.