Ten years of PyPy

Re:Ten years of PyPy by jamesjporter on Thursday February 28, 2013 @05:06PM (#5301402)

I'm personally very excited about the prospects for NumPy in PyPy, as this is my main use case for python. Relatedly, on the PyPy homepage, you can give money to help grease the wheels on development of specific PyPy features: http://pypy.org/. There's also quite detailed proposals that describe where the money will go, how it will help each feature's development, why each feature is useful/cool, etc. To me this seems like a really awesome way to organize donations for an open source project and gives me greater confidence that my donation will have an impact.

[ Parent | on HN ]

Re:Ten years of PyPy by synparb on Thursday February 28, 2013 @08:57PM (#5302391)

You might also check out numba (http://numba.pydata.org/), which still feels slightly beta, but can speed up numpy code quite a bit with very little boilerplate. I've started experimenting with it as an alternative to my usual approach which is to write or wrap code with Cython. I'm curious to see where PyPy goes with numpy and cffi though.

[ Parent | on HN ]

Re:Ten years of PyPy by hyperbovine on Thursday February 28, 2013 @05:28PM (#5301549)

I am excited too, but I also wonder how much faster it's really going to be. Synthetic benchmarks are great, but I didn't see much of a speedup when I switched to PyPy for some of my real-world code.

[ Parent | on HN ]

Re:Ten years of PyPy by fijal on Thursday February 28, 2013 @05:38PM (#5301594)

You should complain. There is a myriad of reasons why PyPy won't go very fast, for example you're using C extensions. Usually what happens if you have code where parts go insanely fast and parts go insanely slow. Once you identify it, sometimes it is just replacing the library (like pgsql2 -> pgsql2cffi or so) and stuff flies. But it all depends on your workload and the stack a lot. R&D is typically needed to get good speedups. For most RW examples, 2-10x are to be expected.

Shameless plug - if your stuff is not open source and you can't be bothered to do the profiling yourself, there is typically an option to hire someone to do it for you. Get in touch.

[ Parent | on HN ]

Re:Ten years of PyPy by legutierr on Friday March 01, 2013 @06:56AM (#5303826)

Could you explain what you meant when you wrote:

> Another option we tried was using RPython to write CPython C extensions. Again, it turned out RPython is a bad language and instead we made a fast JIT, so you don't have to write C extensions.

On the surface it would seem that using RPython would be a boon to authors of C extensions, so this surprises me. It is also kind of shocking to hear the authors of such a great project basically condemn the language that it was written in (and really, the original raison d'etre of the project itself) as "bad".

[ Parent | on HN ]

Re:Ten years of PyPy by fijal on Friday March 01, 2013 @07:57AM (#5303994)

Whether RPython is a good or a bad language is a subjective fact and it should not be related to the fact whether I wrote it or not right? It's good for what it does (writing VMs), or at least better than writing them in C++ including JIT by hand, but it's much worse than Python. Hence using it for a general purpose project is a bad bad idea and you should use Python instead.

[ Parent | on HN ]

Re:Ten years of PyPy by zwegner on Thursday February 28, 2013 @09:56PM (#5302590)

Same here--I spent a couple days porting a compiler frontend from Python 3->2 just to try out PyPy, and got rewarded with a 2-3x slowdown. I messed around with some tracing parameters, but didn't get too much benefit IIRC.

Of course, this was a couple years ago, and I don't have access to that codebase anymore. Things might've changed since then...

[ Parent | on HN ]

Re:Ten years of PyPy by rplnt on Friday March 01, 2013 @07:19AM (#5303882)

When I first tried PyPy I got pretty bad slowdown as well. Apparently PyPy wasn't really good with long strings.

[ Parent | on HN ]

Re:Ten years of PyPy by edwkar on Thursday February 28, 2013 @10:16PM (#5302654)

I'm using PyPy for real-world code every day, because it runs fast as^H^H^H^H, uhm, very fast! I regularly see 6-15X speedups compared to CPython.

That being said, if your code is mostly IO-bound or calls into foreign-language libraries, PyPy's speedup of the Python code of course won't help much with your wall time.

[ Parent | on HN ]

Re:Ten years of PyPy by dagss on Friday March 01, 2013 @04:16AM (#5303482)

Note that for for-loop-style number crunching code, the speedup you are aiming for over CPython is on the order of 50x-750x depending on cache locality.

I'm definitely not saying that PyPy won't do better than 15x speedup on number crunching code, just pointing out that in the context of NumPyPy a 15x speedup is not very relevant if you want PyPy to be a viable alternative to, say, Julia.

(Disclaimer: I'm a Cython dev)

[ Parent | on HN ]

Re:Ten years of PyPy by illumen on Friday March 01, 2013 @06:05AM (#5303722)

Yeah. It depends on the hardware and problem. A 50x speed up with optimised routines(memory optimized, branch reduced, using SIMD etc) multiplied by 8 cores gives you a total of 400 times speed up. This is what I've seen in real life code I've made. Also, if you offload to some other processor (GPU or dedicated hardware) then of course you can get even faster (again depending on the hardware and problem).

So pypy speed ups aren't very good in comparison to the best you can achieve using other techniques... but you can mostly use the same tricks in pypy as you can in CPython to get those results there too :)

[ Parent | on HN ]

Re:Ten years of PyPy by fijal on Friday March 01, 2013 @07:59AM (#5303999)

PyPy speedups over CPython in numeric code are in 50-200x ballpark, it really depends what you do. We can certainly do better, but it's within 2x of optimized C for most of it (unless there is a very good vectorization going on).

For stuff that I tried pypy was universally faster than Cython on non-type annotated code and mostly faster on type-annotated code from cython benchmarks.

[ Parent | on HN ]

Re:Ten years of PyPy by synparb on Friday March 01, 2013 @10:20AM (#5304627)

It would be great to see a repo with this set of examples with code for both what is run under PyPy and the type-annotated cython code, and the exact settings that were used.

[ Parent | on HN ]

Re:Ten years of PyPy by fijal on Friday March 01, 2013 @01:18PM (#5305930)

I run stuff I found in the cython repo only (good or bad) benchmarks/ or so. It was also ages ago so treat it with a grain of salt. My point is that there is no fundamental reason why cython should be faster than pypy (even with type annotations), because type annotations are essentially done during JITting. In fact, pypy should be faster because of other things, like a faster GC or objects.

[ Parent | on HN ]