News for hackers. Stuff that matters.
Re:NumPy 2.0.0 by scoresmoke on Sunday June 16, 2024 @05:02PM (#40700079)
The most important changes are deprecations of certain public APIs: https://numpy.org/devdocs/release/2.0.0-notes.html#deprecati...

One new interesting feature, though, is the support for string routines: https://numpy.org/devdocs/reference/routines.strings.html#mo...

Re:NumPy 2.0.0 by amelius on Sunday June 16, 2024 @05:45PM (#40700377)
> One new interesting feature, though, is the support for string routines

Sounds almost like they're building a language inside a language.

Re:NumPy 2.0.0 by ssahoo on Sunday June 16, 2024 @06:09PM (#40700528)
No. Native python ops in string suck in performance. String support is absolutely interesting and will enable abstractions for many NLP and LLM use cases without writing native C extensions.
Re:NumPy 2.0.0 by ayhanfuat on Sunday June 16, 2024 @07:23PM (#40700940)
> Native python ops in string suck in performance.

That’s not true? Python string implementation is very optimized, probably have similar performance to C.

Re:NumPy 2.0.0 by csjh on Sunday June 16, 2024 @08:06PM (#40701170)
stringzilla[1] has 10x perf on some string operations - maybe they don't suck, but there's definitely room for improvement

[1] - https://github.com/ashvardanian/StringZilla?tab=readme-ov-fi...

Re:NumPy 2.0.0 by mirashii on Monday June 17, 2024 @12:11AM (#40702241)
It is absolutely true that there is massive amounts of room for performance improvements for Python strings and that performance is generally subpar due to implementation decisions/restrictions.

Strings are immutable, so no efficient truncation, concatenation, or modifications of any time, you're always reallocating.

There's no native support for a view of string, so operations like iteration over windows or ranges have to allocate or throw away all the string abstractions.

By nature of how the interpreter stores objects, Strings are always going to have an extra level of indirection compared to what you can do with a language like C.

Python strings have multiple potential underlying representations, and thus have some overhead for managing and dealing with those multiple representations without exposing those details to user code

Re:NumPy 2.0.0 by Too on Monday June 17, 2024 @12:55AM (#40702424)
There is a built in memoryview. But it only works on bytes or other objects supporting the buffer protocol, not on strings.
Re:NumPy 2.0.0 by bvrmn on Monday June 17, 2024 @04:46AM (#40703581)
For numpy applications you have to always box a value to get a new python string. It quite far from fast.
Re:NumPy 2.0.0 by topper-123 on Sunday June 16, 2024 @07:00PM (#40700806)
Yeah, operating on strings has historically been a major weak point of Numpy's. I'm looking forward seeing benchmarks for the new implementation.
Re:NumPy 2.0.0 by nerdponx on Sunday June 16, 2024 @10:25PM (#40701826)
It's already very much a DSL, and has been for the decade-ish that I've used it.

They're not building a language. They're carefully adding a newly-in-demand feature to a mature, already-built language.

Re:NumPy 2.0.0 by ahurmazda on Sunday June 16, 2024 @07:24PM (#40700941)
This one will be rough :|

> arange’s start argument is positional-only

Re:NumPy 2.0.0 by haiguise on Monday June 17, 2024 @06:03AM (#40703994)
Looks like that might get reverted [0].

[0] https://github.com/numpy/numpy/pull/25955

Re:NumPy 2.0.0 by brcmthrowaway on Sunday June 16, 2024 @07:07PM (#40700851)
Does numpy use GPU?
Re:NumPy 2.0.0 by ahurmazda on Sunday June 16, 2024 @07:22PM (#40700935)
No.

You may want to check out cupy

https://cupy.dev/

Re:NumPy 2.0.0 by etbebl on Sunday June 16, 2024 @09:45PM (#40701612)
Interesting that the new string library mirrors the introduction of variable-length string arrays in Matlab in 2016 (https://www.mathworks.com/help/matlab/ref/string.html).