News for hackers. Stuff that matters.
Re:How fast can we make interpreted Python? by ProblemFactory on Saturday July 27, 2013 @12:35PM (#6113493)
On the other hand, having a stable C extension API is what allows Numpy and Cython (http://docs.cython.org/src/quickstart/cythonize.html) to be as good and fast as they are.

My experience with numerical algorithms in Python has been a 10000x speedup and 10x memory reduction by using Numpy arrays and C/fortran functions. And when Numpy doesn't have a necessary function, a few extra lines of static C types and on-import compilation with Cython has still resulted in a 1000x speedup.

"Optimizing Python" needs to look at the whole existing ecosystem of C libraries - to be useful the end result has to be faster than what's possible now, not just faster than pure Python.

Re:How fast can we make interpreted Python? by tomrod on Saturday July 27, 2013 @02:28PM (#6113870)
What exactly is Cython? I've not had a reason to use it yet; are the speed improvements worth it?
Re:How fast can we make interpreted Python? by ProblemFactory on Saturday July 27, 2013 @03:24PM (#6114077)
Cython is a runtime Python-to-C compiler.

The speed improvements are worth it given how easy it is to use, I have seen literally a 1000x speedup on a time-series data analysis script from adding about 10 lines of static type annotations.

The functionality I like best about it are:

* It can be used transparently without any makefiles or compilation steps. Add "import pyximport; pyximport.install()" at the top of your main script, and now every imported Cython-capable module is compiled on the fly at runtime.

* You can start out with no changes to your python modules. All libraries and features still work within compiled modules. Then you can slowly start adding static types to a few variables at a time. The annotated variables become very fast native C integers/doubles/functions, instead of Python objects.

Re:How fast can we make interpreted Python? by tomrod on Saturday July 27, 2013 @04:20PM (#6114279)
Sounds intriguing. I know a fair bit of C, python, Fortran. I've used f2py a bit. Do you have any good Cython tutorial you like to recommend?
Re:How fast can we make interpreted Python? by ProblemFactory on Saturday July 27, 2013 @04:33PM (#6114334)
For my use cases, the standard documentation and tutorials (http://docs.cython.org/index.html) have been enough.

* Install it with pip, the packages in your OS distribution may be out of date,

* Rename your numeric code module's .py file to .pyx,

* Use pyximport from the main script ( http://docs.cython.org/src/userguide/source_files_and_compil...) to have it compiled at runtime without any extra build steps, and then

* Start experimenting with adding a few "cdef"s (http://docs.cython.org/src/quickstart/cythonize.html and http://docs.cython.org/src/tutorial/numpy.html)