News for hackers. Stuff that matters.

cfbolztereick

Re:"Warn about PyPy being unmaintained" on Tuesday March 10, 2026 @04:06AM (#47320324)
Exactly correct. PyPy is a replacement for CPython 3.11, which aims to be fully compatible with pure Python code (C extensions are a more complicated story).
Re:"Warn about PyPy being unmaintained" on Sunday March 08, 2026 @11:01AM (#47297877)
Timely management of external resources is what the `with` statement has been for since 2006, added in python 2.5 or so. To debug these problems Python has Resource Warnings.

Additionally, CPython's gc is also only eager in a best effort kind of way. If cycles are involved it can take long to release memory. This will become even more the case in future versions of CPython, in the free threading variants.

Re:"Warn about PyPy being unmaintained" on Sunday March 08, 2026 @04:15AM (#47295551)
PyPy isn't unmaintained. We are certainly fixing bugs and are occasionally improving the jit. However, the remaining core devs (me among them) don't have the capacity to keep up with cpython. So for supporting new cpython versions we'll need new people to step up. For 3.12 this has started, we have a new contributor who is pushing this along.
Re:The Awful German Language on Friday May 16, 2025 @04:15PM (#44009416)
The English term for 'Bildungsroman' is actually... 'bildungsroman' https://en.wikipedia.org/wiki/Bildungsroman
Complaining about a rest (however long) in a piece by the composer of 4'33'' is certainly A Take.
Re:The Prospero Challenge on Monday March 24, 2025 @04:12PM (#43464953)
This is really fun, I'm getting badly nerd sniped by this.
Re:Musings on Tracing in PyPy on Monday January 13, 2025 @09:14AM (#42683601)
It should get pretty good performance yes. Not sure how native like we get with the jit. Gut feeling would be a bit slower than gcc -O0? I would be very interested in your experience if you do try it.
Re:Musings on Tracing in PyPy on Saturday January 11, 2025 @03:01AM (#42664192)
The equivalent of this (and strongly inspired by luajit's ffi) in the python world is cffi, btw: https://cffi.readthedocs.io/en/stable/
Re:Musings on Tracing in PyPy on Friday January 10, 2025 @02:18PM (#42659006)
And a significant chunk of these important extension libraries are supported in pypy using the emulated C-extension api (cpyext)
Yeah, that's exactly right. It's operating on an SSA-based intermediate representation. So if an operation gets removed, the arguments are still being computed. The earlier operations that produce those arguments can also be removed by dead code elimination (which runs later), but indeed only if they have no side effects.

I suppose the notation of the rule heads makes it look like there actually are expression trees, which is maybe sort of confusing.

I only support integer operations in the DSL so far.

(but yes, turning the python expression x*y+z into an fma call would not be a legal optimization for the jit anyway. And Z3 would rightfully complain. The results must be bitwise identical before and after optimization)

Re:Making PyPy's GC and JIT produce a sound [video] on Thursday August 22, 2024 @03:54PM (#41323852)
Yeah, that's a good point!

The 'empty' parts of the beginning of the program are actually a 'configure' phase, where the program runs a bunch of subprocesses (C compiler with small query programs) and the vm is mostly waiting for the results of that. That's why there are pauses in the coins.

On the other end, I also artificially limit the samples to 10 coins a second (iirc), because if the jit is really compiling a lot it would be just too annoying.

Re:Making PyPy's GC and JIT produce a sound [video] on Thursday August 22, 2024 @03:50PM (#41323827)
Good idea! There are various other extensions possible.

Unfortunately there's a bit of an observer effect, the sounds are made with python code running in the same process, and the more complicated I made the sound generation logic, the more garbage it produces itself .

Re:Making PyPy's GC and JIT produce a sound [video] on Thursday August 22, 2024 @03:48PM (#41323801)
Oh wow, that's just amazing, thanks for the link! From the article:

''' Interestingly, there was a technical bump or glitch in the Unity Engine. A feature that no user would normally have any control over, that caused a bump that made it all the way from the D23 footage to the final film. Every once in awhile the Unity Engine clears our any unneeded data or assets from the Engine.

Unfortunately, when this random function, deep in the code called ‘garbage collection’ ran it could cause a tiny pause in the smooth movement of the master Unity Camera. One such ‘bump’ happens in a shot in the D23 trailer. After the trailer was complete, the team discovered what the issue was and fixed it. But even when the shot was redone much later, this ‘bump’ is still in the final camera move, just because Rob Legato liked the feel of the recorded natural move, even with the bump. For the creative team, ironically given its causes, it just felt natural. '''

https://www.fxguide.com/fxfeatured/how-virtual-production-wo...

Re:Mining JIT traces for missing optimizations with Z3 on Monday July 22, 2024 @05:18AM (#41032336)
Yep, typo, thanks for pointing it out! I'm fixing it.
Re:Mining JIT traces for missing optimizations with Z3 on Sunday July 21, 2024 @05:12PM (#41028161)
PyPy has formally verified the integer abstract domain using Z3, a quite important part of our jit optimizer (will write about that in the coming weeks).

We also run a fuzzer regularly to find optimization bugs, using Z3 as a correctness check:

https://pypy.org/posts/2022/12/jit-bug-finding-smt-fuzzing.h...

The peephole optimizations aren't themselves formally verified completely yet. We've verified the very simplest rules, and some of the newer complicated ones, but not systematically all of them. I plan to work on fully and automatically verifying all integer optimizations in the next year or so. But we'll see, I'll need to find students and/or money.

Re:Mining JIT traces for missing optimizations with Z3 on Sunday July 21, 2024 @01:52PM (#41026750)
All the minimized inefficiencies that are found by my script are already optimizations. They just happen to be rather specific patterns, so they need to be suitably generalized. There's another Regehr etal paper about how to do that automatically: https://dl.acm.org/doi/10.1145/3649837

(llm's aren't involved, it's all based on z3)

I don't plan on implementing something like this for now, I'd rather take the inefficiencies and manually extract optimizations out of them and implement them in PyPy's jit.

Re:PyPy has moved to Git, GitHub on Tuesday January 02, 2024 @02:20PM (#38845770)
PyPy was started early in 2003 too, the first release took a while. PyPI was branded as 'The Cheeseshop' in the early years.
Yeah, that's my current project, already found a bunch of missing small features in the optimizers.
There would be an actual miscalculation, yes. which is a really annoying kind of bug, also because these jit bugs tend to go away if you turn the debugger on, making them even more frustrating to track down.

The ones we found so far are all extremely unlikely to occur in 'regular' Python code, because they require the use of internal pypy specific numerical operations.

Yeah, the generated traces are 'sensible' by construction (they don't compute anything sensible, but they are well-formed).