News for hackers. Stuff that matters.
Re:Four MLs (and a Python) by andor on Thursday April 30, 2015 @05:09AM (#9464000)
It completely depends on the application. I have yet to see a use case that benefits from PyPy. Here's an example I tried with CPython2, 3 and PyPy yesterday: It loads the most probable translations for all unigrams from a Moses phrase translation table (11GB) into a dict, and using that dict translates 73 MB of text word by word. Most of the time is spent filling the dict/reading the large file.

  $ time python literal_translation.py phrase-table en test4
  translation table contains 76885 elements
  ...................................................................... 070
  .......................................................
  125 files translated
  
  real	1m14.345s
  user	1m9.836s
  sys	0m4.157s
  
  
  $ time python3 literal_translation.py phrase-table en test5
  translation table contains 76885 elements
  ...................................................................... 070
  .......................................................
  125 files translated
  
  real	2m17.680s
  user	2m2.605s
  sys	0m4.987s
  
  
  $ time pypy literal_translation.py phrase-table en test6
  translation table contains 76885 elements
  ...................................................................... 070
  .......................................................
  125 files translated
  
  real	1m13.347s
  user	1m0.033s
  sys	0m4.734s

https://gist.github.com/andreasf/af6bdc00cf5a928712b5
Re:Four MLs (and a Python) by 314 on Thursday April 30, 2015 @06:38AM (#9464223)
If your problem is memory-bound then no compiler will help you.

Memory-mapping a serialisation of the dict would help, but I'm not sure how you would do that outside of an extension lib.

Re:Four MLs (and a Python) by unhammer on Thursday April 30, 2015 @07:05AM (#9464295)
1m for 11GB sounds IO-bound, but yeah that's still not a use-case where switching to pypy will make a difference.

But did you try using it as a service (ie. avoid the startup time) and timing the actual translation?

At work, we switched to pypy for the speed. For a spell-checker I made in python, pypy made the dictionary compiling 3x faster and the spelling 2x faster, so it can really matter. On the other hand, pypy's memory usage is often a bit worse, and startup time can also be a bit slower, at least if your program has an otherwise "zero" startup time.

Re:Four MLs (and a Python) by andor on Thursday April 30, 2015 @07:36AM (#9464395)
No, I didn't try to optimize this any further. I just checked whether any of the new interpreters is really faster. It's a simple one-off experiment, and compared to real statistical machine translation with Moses on the same data the translation is pretty much instant. Moses needs something on the order of 1600 CPU-hours (50 nodes * 8 threads * 4 hours).
Re:Four MLs (and a Python) by syllogism on Thursday April 30, 2015 @08:32AM (#9464622)
NLP stuff tends to be more data-bound, which PyPy can't really help with.

If it's worth the effort, use Cython, design your data structures carefully, and you'll get C level performance.

Re:Four MLs (and a Python) by andor on Friday May 01, 2015 @08:09AM (#9470772)
Hey, thanks for spaCy! It's a great library.
Re:Four MLs (and a Python) by zo1 on Thursday April 30, 2015 @02:09PM (#9466664)
This isn't an isolated test, and tells us almost nothing. Because you've clumped together IO as well as the bulk of the actual processing work.

Rather, when you do the test, only test the "mapping" part, and only test the "IO" part. That way you can clearly see the difference between the two.

Re:Four MLs (and a Python) by GFK_of_xmaspast on Thursday April 30, 2015 @08:22AM (#9464575)
I notice python3 is twice as slow.
Re:Four MLs (and a Python) by andor on Thursday April 30, 2015 @07:45AM (#9464426)
To whoever downvoted this -- how about posting a more comprehensive set of benchmarks instead? My single benchmark is not in any way less meaningful than the parents. But as I said, I often try PyPy on longer-running Python code, and so far saw no significant gains.
Re:Four MLs (and a Python) by fijal on Thursday April 30, 2015 @09:52AM (#9465020)
Your benchmark is terrible - it's impossible to run it because you don't provide a dataset that you used (or I'm missing some description). It also stresses one aspect of the language (processing strings, in a very specific way) and yet you claim this is representative.

However, we're more than happy to consider it a pypy bug if we're not faster than CPython. Please provide clearer instructions how to reproduce it and ideally also other benchmarks that did not work for you.

Re:Four MLs (and a Python) by andor on Friday May 01, 2015 @01:36PM (#9473117)
Great, next time I'll hit your bug tracker directly ;-)

The script and data are available here: http://zentrale1.com/~an/literal-translation.tar.xz

The texts were scraped from the European Parliament web site. The phrase table was built with Moses from the Europarl parallel corpus [0] minus the texts.

The script itself was supposed the be baseline input for another experiment, basically it's the worst way of translating something i could think of.

[0] http://statmt.org/europarl/

Re:Four MLs (and a Python) by fijal on Monday May 04, 2015 @06:21AM (#9485273)
can you please provide me the data in somehow reproducible form (I have no clue how to use all the tools you mentioned for example), ideally as a .tar.bz2 file somewhere I can download and run without any prior knowledge. I'm happy to improve pypy for that particular use case.
Re:Four MLs (and a Python) by brendano on Thursday April 30, 2015 @01:22PM (#9466356)
"Terrible" is a little harsh. I found the benchmark to be interesting because it's similar to many NLP-style analysis tasks I have myself. Of course reproducible ones would be great too.