News for hackers. Stuff that matters.

steve_s

Limited C API is not as abstract as HPy. Most notably Limited C API still exposes reference counting as memory management mechanism, HPy abstracts that. However, ecosystem wide adoption of limited C API and stable ABI would already improve things significantly.
There is basic GraalPy support in Maturin[0] and PyO3[1], the problem is often that packages require older Maturin/PyO3 versions and/or they use CPython-isms, semi-public APIs, etc., but it is getting better, for example [2].

It is fair to say that large projects with a huge set of dependencies will likely face some compatibility issues, but we're working on ironing this out. There is GraalPy support in setup-python GitHub action. GraalPy is supported in the manylinux image [3]. Hopefully soon also in cibuildwheel [4].

[0] https://github.com/PyO3/maturin/pull/1645 (merged)

[1] https://github.com/PyO3/pyo3/pull/3247 (merged)

[2] https://github.com/pydantic/jiter/pull/135 (merged)

[3] https://github.com/pypa/manylinux/pull/1520 (merged)

[4] https://github.com/pypa/cibuildwheel/pull/1538

Re:Statement on CVE-2024-27322 on Thursday May 23, 2024 @04:25AM (#40452032)
RDS files are a common way of sharing serialized R objects. Promises are valid R objects and supported by this serialization format. They always have been and I believe it is an intentional feature. The problem is that some people may think of RDS files as more convenient CSV files, but they are not.
Re:Google lays off its Python team on Monday April 29, 2024 @04:37AM (#40195990)
No, you'd have to take all the profit of all the companies in the country and divide that by the number of citizens. That would be true socialism. Like a kibbutz in Israel. IMO it is as much utopia as 100% free market capitalism. Almost everywhere you have something in between, including in the US.
Re:Faster R with FastR on Thursday October 11, 2018 @04:13PM (#18196694)
FastR doesn't alter the semantics of R, so when dplyr copies a vector in GNU-R then FastR has to copy it too. However, FastR does use reference counting (not sure if that's turned on in GNU-R 3.5.1 now) so it may avoid some unnecessary copies.
Re:Faster R with FastR on Thursday October 11, 2018 @03:52PM (#18196522)
There is a plot of warm-up curves for this specific example. Search for "To make the analysis of that benchmark complete, here is a plot with warm-up curves".

However, it is true that the warm-up and memory usage are something we need to improve. We're working on providing native image [1] of FastR. With that, both the warm-up and memory usage shold get close to GNU-R.

[1] https://www.graalvm.org/docs/reference-manual/aot-compilatio...

Re:Faster R with FastR on Thursday October 11, 2018 @03:45PM (#18196465)
Plese open an issue on GitHub if you encounter any more problems with gsub or anything else.
Re:Faster R with FastR on Thursday October 11, 2018 @03:42PM (#18196443)
FastR can actually run all tests of the development version of dplyr with a simple patch. We're working on removing the need for that patch altogether.

data.table is a different beast and we will probably provide and maintain patched version for FastR. They do things like casting data of internal R structure to byte array and then memcopy it to another R structure. This is very tricky to emulate if your data structures actually live on Java side and you're handing out only some handles to the native code.

Re:Faster R with FastR on Thursday October 11, 2018 @03:36PM (#18196388)
FastR also uses native BLAS and LAPACK libraries. It should be possible to link it with Intel MKL as well.

We didn't want to include comparison to R-3.5.X, because FastR itself is based on the base library of 3.4.0, but the results for GNU-R 3.5.1 almost the same as for R-3.4.0.

AFAIK ALTREP is not used that much yet inside GNU-R itself. They can now do efficient integer sequences (i.e. 1:1000 does not allocate 1000 integers unless necessary), which would save a little bit of memory in this example, but that's about it. FastR also plans to implement the ALTREP interface for packages. Internally, we've been already using things like compact sequences.