News for hackers. Stuff that matters.
Re:Tracking Down a Python Memory Leak by bbernard on Wednesday December 07, 2016 @02:41PM (#13125527)
I've had quite a few issues with cPickle myself, so I see what you mean.

Indeed, Python packages built over C extensions can be quite hard to debug, as seen with lxml. But what makes it even harder is the fact that lxml is partly built using Cython... so you deal with Python code, C code generated with Cython, and pure C code (libxml2).

Re:Tracking Down a Python Memory Leak by dom0 on Wednesday December 07, 2016 @07:04PM (#13127388)
While Cython - in my experience - doesn't have many bugs, it still has some, and can, at times, generate blaringly wrong C code (iirc one simple example: slice assigmnent to a uint8_t* tries to call a Python runtime function on the uint8_t* as a PyObject*).
Re:Tracking Down a Python Memory Leak by bbernard on Wednesday December 07, 2016 @07:34PM (#13127513)
Good to know! My only experience so far with Cython comes from lxml. It's a weird language that seems to have a lot of corner cases, though. Just like C++/CLI.
Re:Tracking Down a Python Memory Leak by dom0 on Wednesday December 07, 2016 @07:35PM (#13127520)
Yes. Yes it does. Also, because it works on two type systems it has many cases were you'll want to take a look at the generated C code to verify that the "cheap path" was taken and no intermediary Python objects are constructed or Py operators are used -- if performance matters, that is.

On the other hand it is a radically simpler way to write bindings that also contain logic, or to write rather fast code without straying to far from Python. Plus, it can cythonize almost all code, even very dynamic code with closures, which will still often improve performance on it's own (no interpreter, but still Python runtime for every op). And that is then a nice base to do further optimizations.