News for hackers. Stuff that matters.

ngoldbaum

ngoldbaum on Hacker News

Re:Python 3.15.0 on Friday October 09, 2026 @02:57PM (#50025147)
It should be good to go, but of course I don’t know what your codebase looks like and issues are always possible in any code, “good to go” or no. Please do give it a try and try testing on 3.15 and 3.15t.
Re:Python 3.15.0 on Friday October 09, 2026 @02:56PM (#50025138)
I’m glad to see there’s ongoing work to get protobuf working!

I doubt PyUnstable APIs will get promoted directly to the limited API without at least a release or two in the regular version-specific API first.

That said, if you would be willing to start a thread on discourse describing your need and use-case, that is probably the first step to stabilizing the APIs you want. It may also turn out there’s an alternate way to get protobuf working.

Please do feel free to reach out privately. I’d love to be able to chat with people working on protobuf and grpcio, they come up reasonably often and it’s hard to get official updates from inside Google.

Re:Python 3.15.0 on Friday October 09, 2026 @11:36AM (#50022069)
My headline feature is the new “abi3t” stable ABI for the free-threaded build. While Petr Viktorin did most of the CPython implementation, I’ve been trying to make sure ecosystem support is ready. It’s been a rewarding but quite challenging project to make sure everything is working. There were some late nights leading up to the beta1 release when we found a Windows-specific issue that needed a fix.

I’m particularly proud that the cryptography project is already shipping a single abi3.abi3t wheel for each platform on Python 3.15 or newer. The GIL-enabled build and free-threaded build can both use the same wheel now, because PyObject is opaque.

If you want to learn more about this, I gave a talk at EuroPython this year on Python’s ABI and the road to building and releasing abi3t today. See https://youtu.be/An8lO29SxXE.

Re:Scaling NumPy on Free-Threaded Python on Friday August 07, 2026 @09:07AM (#49209835)
I’m saying that it’s a new issue on the free-threaded build.
Re:Scaling NumPy on Free-Threaded Python on Thursday August 06, 2026 @07:14AM (#49195178)
There’s also the fact that INCREF and DECREF on shared objects is a lot more expensive than plain integer addition, so stuff becomes a bottleneck that was never a bottleneck. Kumar also fixed a bottleneck caused by a lock added only for safety on the free-threaded build. It’s hard to tell in advance than a fancy lock-free data structure is needed for something.
Re:Scaling NumPy on Free-Threaded Python on Thursday August 06, 2026 @07:11AM (#49195148)
I’m not sure why it took me, a NumPy developer, looking at the benchmark numbers and saying “hmm, this is a bug”. But that is what it took. People are sometimes slow to treat behavior in dependencies like NumPy as bugs.
Re:What Every Python Developer Should Know About the CPython ABI on Sunday July 26, 2026 @01:22PM (#49060220)
I’m glad you enjoyed the post. It really helped me to crystallize my understanding. I was also confused about the API/ABI distinction for a long time.
Re:What Every Python Developer Should Know About the CPython ABI on Sunday July 26, 2026 @01:17PM (#49060171)
Thanks for pointing that out, I’ll fix that.
Re:GitHub is (again) having issues on Tuesday August 12, 2025 @11:27AM (#44877560)
I wonder why the github status page has an atlassian cookie request pop-up.
Re:Ty: A fast Python type checker and language server on Thursday May 08, 2025 @10:46AM (#43926553)
Either something about beanie babies or something riffing on "thank you". Couldn't ever make up my mind then basically forgot about it.
Re:Ty: A fast Python type checker and language server on Wednesday May 07, 2025 @04:10PM (#43920112)
I gave away the “ty” on pypi to Astral week or so ago. I wanted to use it for a joke a few years ago but this is a much better use for a two letter project name. They agreed to make a donation to the PSF to demonstrate their gratefulness.
Re:Branchless UTF-8 Encoding on Friday January 17, 2025 @04:47PM (#42743614)
You could do two passes over the string, first get the total length in bytes, then fill it in codepoint by codepoint.

You could also pessimistically over-allocate assuming four bytes per character and then resize afterwards.

With the API in the linked blog post it's up to the user to decide how they want to use the output [u8;4] array.

Re:My NumPy Year: Creating a DType for the Next Generation of Scientif... on Thursday October 24, 2024 @09:43AM (#41935414)
The main difference is the strings are stored in a single contiguous arena buffer (with some minor caveats if you mutate the array in-place). With object strings each string has its own heap allocation.

More details in NEP 55: https://numpy.org/neps/nep-0055-string_dtype.html

This post is based on the content of a 25 minute talk and it’s hard to explain everything fully…

Re:My NumPy Year: Creating a DType for the Next Generation of Scientif... on Thursday October 24, 2024 @09:40AM (#41935377)
Ah I could see how that’s confusing. I was trying to indicate that the size stored for the string in the example is 28, but it’s stored in a 64 bit uint.
Re:My NumPy year: Creating a DType for the next generation of scientif... on Thursday October 24, 2024 @09:37AM (#41935353)
They’re stored on the DType instance. This requires that there’s only one DType instance per “owned” array buffer, which I figured out how to do along with Sebastian Berg and others using the new DType system.
Re:My NumPy year: Creating a DType for the next generation of scientif... on Wednesday October 23, 2024 @08:23PM (#41930637)
That is something I’d like to see but I don’t want to wade into the already very complicated discussion around arrow strings in pandas. If a Pandas developer wanted to take this on I think that would make things easier since there’s so much complexity around strings in Pandas.

That said there is a branch that gets most of the way there: https://github.com/pandas-dev/pandas/pull/58578. The remaining challenges are mostly around getting consensus around how to introduce this change.

If NumPy had StringDType in 2019 instead of 2024 I think Pandas might have had an easier time. Sadly the timing didn’t quite work out.

Re:My NumPy year: Creating a DType for the next generation of scientif... on Wednesday October 23, 2024 @04:41PM (#41929095)
This was a case of convergent evolution, both projects ended up working simultaneously on similar ideas.

One issue with using Arrow directly in NumPy is PyArrow exposes an immutable 1D array, while NumPy exposes a mutable ND array.

See also https://numpy.org/neps/nep-0055-string_dtype.html#related-wo...

Re:My NumPy year: Creating a DType for the next generation of scientif... on Wednesday October 23, 2024 @04:39PM (#41929082)
Well, there was no concept of sidecar storage. Now we have the hack we came up with for StringDType to store data on the DType instance and also make it so StringDType arrays don't share StringDType instances, unless the array is a view.

EDIT: looking back at the NEP, I'm not sure it does a great job explaining exactly how the per-array descriptor works. Ultimately it's powered by a hook in the DType API: https://github.com/numpy/numpy/pull/24988. There is only one spot in NumPy where array buffers are allocated, so we hooked there and made sure any arrays with newly allocated buffers get a new DType instance.

Re:My NumPy year: Creating a DType for the next generation of scientif... on Wednesday October 23, 2024 @03:34PM (#41928560)
Thank you this really means a lot.
Re:The Fastest Mutexes on Wednesday October 02, 2024 @08:40PM (#41726256)
No, it wouldn’t make sense to. Use threading.Lock for that. PyMutex is available in the CPython C API.
Re:The Fastest Mutexes on Wednesday October 02, 2024 @03:40PM (#41724323)
https://github.com/numpy/numpy/issues/26510#issuecomment-229...

And now that I look at that again I realize I forgot to finish that up!

Re:The Fastest Mutexes on Wednesday October 02, 2024 @02:52PM (#41723866)
This style of mutex will also power PyMutex in Python 3.13. I have real-world benchmarks showing how much faster PyMutex is than the old PyThread_type_lock that was available before 3.13.
Re:Anaconda sued Intel for copyright infringement over AI software on Friday August 09, 2024 @04:24PM (#41205124)
The complaint [1] is linked in the article. They're suing over Intel internally using and redistributing the anaconda software distribution.

[1] https://fingfx.thomsonreuters.com/gfx/legaldocs/znvnxomagvl/...

Older »