News for hackers. Stuff that matters.
Re:Python C API, PyPy and the road into the future by binarycrusader on Monday November 16, 2015 @08:53PM (#10578705)
I strongly disagree with the author's assertions that the need for better C API support is strictly a need for integrating with "legacy applications".

The fact of the matter is that a vast array of systems-level software and high-performance software is only available via a C API interface. New or old, that's still true today.

I've used ctypes and I've used cffi; I'm glad they exist, but authors of popular packages often don't have the luxury of supporting both of them and the CPython API.

If by legacy, they're attempting to paint the CPython API as the legacy one, I also find that a misleading argument at best. PyPy is not yet the official future of Python, and attempting to paint themselves as the successor without any clear indicator of support for that future from the Python leadership seems odd.

I'm all for them getting financial support they need to do the work, but attempting to justify that work by claiming only "legacy" applications need it seems uninformed at best.

Re:Python C API, PyPy and the road into the future by Animats on Monday November 16, 2015 @10:45PM (#10579042)
"PyPy is not yet the official future of Python."

We're past that point. PyPy is the future. CPython is the past. The PyPy team succeeded in making an optimizing compiler for a language that fought them every step of the way with gratuitous hidden dynamism. That's a considerable achievement. It extends the life of Python by making it competitive on speed.

Go exists mostly because Python was too slow. Google used to use Python quite a bit internally, but their effort to speed it up, Unladen Swallow, was a disaster. That provided some of the motivation for Go.

Re:Python C API, PyPy and the road into the future by Demiurge on Monday November 16, 2015 @11:19PM (#10579148)
> We're past that point. PyPy is the future. CPython is the past.

CPython is the past and is the present, and the future you can't predict. However, if you try to extrapolate, you should perhaps consider as much historical context as possible. For instance, why is Python what it is? Is it speed? I think it is the accessibility of language syntax, design and features. CPython is the base of this language evolution, and PyPy is improving just speed. So I would extrapolate CPython to always be more popular.

Go exists because someone at Google wanted to make better C and C++, a statically typed language. It doesn't have much to do with Python. Google always preferred C++ and Java to python because of static typing, not just because of speed.

Overall, I think it's a mistake to fixate so much on speed of execution, when often times speed of development is considered more important. This niche is never going away, despite of how hard some people hammer square pegs in round holes.

Re:Python C API, PyPy and the road into the future by BuckRogers on Monday November 16, 2015 @11:37PM (#10579192)
>Overall, I think it's a mistake to fixate so much on speed of execution, when often times speed of development is considered more important. This niche is never going away, despite of how hard some people hammer square pegs in round holes.

That's the point to PyPy. You get fast speed of development and fast CPU performance. Best of every world. That's why we use it and not CPython. It's already bigger than you think.

Re:Python C API, PyPy and the road into the future by Demiurge on Monday November 16, 2015 @11:48PM (#10579228)
>That's the point to PyPy. You get fast speed of development and fast CPU performance. Best of every world. That's why we use it and not CPython. It's already bigger than you think.

What you use it for, where you need the speed and don't have a C based module to rely on? I don't mind having a bit more speed for 'free', but every time I tried, pypy hasn't been hassle free due to some module compatibility, and yet pure python code has never been a bottle neck. I have been using Python for +10 years.

Anyway, the above question is for curiosity sake, it doesn't change the point that CPython is where new language features are added. If Guido adopts pypy tomorrow, I would be happy, but otherwise, it will always try to catch up, so I don't see how it can be the future.

Re:Python C API, PyPy and the road into the future by yason on Tuesday November 17, 2015 @04:31AM (#10580034)
I think the best thing about PyPy is that it's a modern implementation freed from the stupid architectural choices that plague CPython, and that PyPy doesn't have to get along with Guido's quirks. By that I mean things like Python 3 which did some things right but most things wrong. It's seven years since Python 3 was released and there still is no pressing need to unilaterally switch. Thus, there's no need to catch up, PyPy can continue the 2.x line while importing useful features from 3.x and possibly adding some of their own (I hope they do).
Re:Python C API, PyPy and the road into the future by BuckRogers on Tuesday November 17, 2015 @01:26AM (#10579477)
>the above question is for curiosity sake, it doesn't change the point that CPython is where new language features are added.

So far. Everything changes the moment the PyPy team announces they're an official continuation of Python2. They can easily integrate more 3rd party backported features from 3->2 into PyPy4. Guido doesn't really matter there.

That said, it doesn't take that to get me to use PyPy. The speed improvement is a dream come true. Writing pure Python is much more preferable to me than writing a C extension thus PyPy is the present and future for most of us.

Re:Python C API, PyPy and the road into the future by Demiurge on Tuesday November 17, 2015 @07:20AM (#10580583)
Well, that would be kind of hostile of PyPy, I really don't see that happening.

I'm still quite curious what is your use case for PyPy where it is such a god send? I don't enjoy C, but it's just been so rare that I had to Cython or C anything, things are quite well optimized in the ecosystem.

Re:Python C API, PyPy and the road into the future by BuckRogers on Tuesday November 17, 2015 @02:28PM (#10583387)
Why would it be hostile? There's tons of Python2 users out there like me that need support and a secure path forward with Python, and my code.

PyPy didn't create that situation, CPython did. They're just filling a market need. It may be opportunistic, but it's hardly hostile. If anything was hostile it was CPython3, but I'd say neither would be "hostile" moves.

Everyone should always act in their own best interests. Especially users like myself. The CPython team did what they felt was in theirs. I do what I feel is mine. PyPy should do the same, not be fearful of some toxic, "hostile" accusation.

The godsend is that I don't have to worry about CPU performance being a limiting factor at all. That's a big deal and reduces my hardware needs to do the same amount of work. My VPS is hardly overpowered, and I like it that way because it's cheap for my projects. :) CPython only exacerbates hardware issues.

It's very nice to have a simple interpreter to fall back on, but at this point in time I think most dynamic languages need to be on a JIT like PyPy. It's just too good and I can't wait for the STM branch to be merged into PyPy4. No more GIL for those of us using it.

Re:Python C API, PyPy and the road into the future by JelteF on Tuesday November 17, 2015 @05:42AM (#10580259)
I know more people that use Python 3 in production than ones that use PyPy in production. If PyPy really thinks of itself as being the future, I don't understand why they are not adding more Python 3 support. I understand that they don't get funding for that, which to me only means that the people that use PyPy have lots of legacy code themselves.

They might be the future of Python 2.7 though, a very fast Python 2.7, but still with all the annoyances and quirks that Python 2.7 brings. And without a lot of the nice features that are in Python 3.

Re:Python C API, PyPy and the road into the future by mangeletti on Monday November 16, 2015 @11:01PM (#10579097)
I upvoted you on the sheer idea that performance is paramount to the future of Python. I think Python needs to be 5x faster and use 1/5th the memory for the same tasks, in order to remain relevant years from now.

But, I still think PyPy has a long way to go.

Re:Python C API, PyPy and the road into the future by mangecoeur on Tuesday November 17, 2015 @05:23AM (#10580199)
CPython is the past only if you conveniently ignore the entire Scientific Python community, which is a huge use case for Python and depends heavily on C-extensions. Frankly saying a transition to PyPy is "done" when NumPy support is not complete is laughable. Huge Python shops (we're talking NASA, CERN, and other institutions on that scale) depend on a Python stack that's heavily reliant on a huge range of things, including the C-API, that aren't even close to mature in PyPy.
Re:Python C API, PyPy and the road into the future by lmm on Tuesday November 17, 2015 @05:34AM (#10580230)
NumPy proves that you can do everything that you'd need to do without the CPython C API, no? The fact that other things used in scientific Python only offer a C API interface and not cffi seems like a "legacy"-shaped problem.
Re:Python C API, PyPy and the road into the future by mangecoeur on Tuesday November 17, 2015 @06:38AM (#10580434)
Only if you label basically everything as "legacy", which is really stretching the definition. Just because someone invented a New Thing, doesn't mean all the Other Things are suddenly "legacy" overnight.

Fact is, PyPy may be "fast", but in science CPython is in effect much faster because you can use the massive and optimised scientific stack including python, pandas, hdf5, etc etc

Re:Python C API, PyPy and the road into the future by DasIch on Monday November 16, 2015 @09:00PM (#10578727)
The author only claims that better CPython C API support is needed for legacy applications.

Of course there are a vast number of applications that have C API themselves that one might want to accesses. You can interact with those through cffi in a way that's simple and performant already.

Re:Python C API, PyPy and the road into the future by bpicolo on Tuesday November 17, 2015 @12:16AM (#10579291)
The bigger issue is the larger number of packages using the C Api, not the applications themselves
Re:Python C API, PyPy and the road into the future by stuaxo on Tuesday November 17, 2015 @03:47AM (#10579895)
GUI apps is an area that comes to mind. Gtk3 uses the C API, although you can try and use PGI it is not fully ready.

In fact, lots of old graphics and sound APIs are the same... basically anything that is not hooked up to the web.

Re:Python C API, PyPy and the road into the future by fijal on Tuesday November 17, 2015 @04:17AM (#10579980)
If you can change the extension, then going for cffi seems like a no-brainer. It works on cpython, it's easier to use, safer etc. Usually the problem is if the application/library is too big/too cumbersome. Then I think it's fair to call it legacy
Re:Python C API, PyPy and the road into the future by frozenport on Tuesday November 17, 2015 @01:40AM (#10579519)
C APIs also offer significantly easier distribution, as you don't need to worry about things like name mangling or support for other languages. Much better than alternatives like COM.
Re:Python C API, PyPy and the road into the future by lucian1900 on Tuesday November 17, 2015 @04:56AM (#10580101)
cffi works fine on CPython and it's also quite fast. It ends up with pretty much the same result.