News for hackers. Stuff that matters.
Re:Data-Oriented Programming in Python by periheli0n on Monday November 28, 2022 @02:05AM (#33770562)
> The paper now consists of 10 Jupyter Notebooks and about 16000 lines of code with 1200 git revisions

I‘ve been there too! And about 1 year after the paper was finally published, fresh starters in my lab couldn’t run the code anymore because of an obscure Pandas error caused by a version change. Then Python 2.7 was shelved and created more problems.

In conclusion the considerable effort I put in to create Jupyter notebooks recreating every single figure from the paper was not worth it.

Re:Data-Oriented Programming in Python by Helmut10001 on Monday November 28, 2022 @02:13AM (#33770588)
I created Docker Containers for every "release" of the paper that we maintain in our Gitlab Registry (a CI builds those images automatically from `.gitlab-ci.yml`, `docker-compose.yml` using `docker:dind`), so you can pull a specific Docker Image for every version of the paper that will definitely have the correct dependencies (because the Jupyter Notebooks were tested with each specific version), including Jupyter itself.
Re:Data-Oriented Programming in Python by ogarten on Monday November 28, 2022 @03:44AM (#33771195)
That is awesome and should be the minimum standard, but even just getting the entire infrastructure set up is something almost all scientists are not willing to do.

In my experience, one should be happy if code is version controlled in a proper way. Too often, it isn't. In ML this might be a little different but in my field at least (electrical engineering) this is not the case at all.

Re:Data-Oriented Programming in Python by Helmut10001 on Monday November 28, 2022 @03:50AM (#33771230)
Yes, I know and I was pretty stubborn with my initial goals to go through with this.. (probably will also cost my employment, but I learned a lot and it was thus still worth it).