News for hackers. Stuff that matters.
Re:Help us collect modern Python NumPy code solutions by johndough on Monday March 28, 2022 @08:20AM (#30829687)
The code quality seems questionable. A few examples:

Repository claims modern code, but uses Python 2 (end of life January 1, 2020) https://github.com/Onelinerhub/onelinerhub/blob/main/python/... Copied from https://wiki.python.org/moin/Powerful%20Python%20One-Liners

Incorrect indentation causing unnecessarily large memory usage https://github.com/Onelinerhub/onelinerhub/blob/main/python/...

Undefined behavior for i = 31 https://github.com/Onelinerhub/onelinerhub/blob/main/c/bits_...

Printing of uninitialized memory https://github.com/Onelinerhub/onelinerhub/blob/main/c/set_v...

Undefined behavior https://github.com/Onelinerhub/onelinerhub/blob/main/c/swap_...

Re:Help us collect modern Python NumPy code solutions by Beltiras on Monday March 28, 2022 @08:36AM (#30829842)
Fixed some of those. Simple case of just pushing the edit button and sending a PR.
Re:Help us collect modern Python NumPy code solutions by johndough on Monday March 28, 2022 @09:23AM (#30830261)
The suggested solution https://github.com/Onelinerhub/onelinerhub/blob/34467e427cc6... still runs out of memory with many files.

I believe the original intention behind using mode="a" was to append to the output file while reading the input files at the same time. This way, there is no need for an ever-growing string array.

But there are still many other issues like using default platform string encoding instead of detecting it properly or at least using utf-8, checking for ".txt" anywhere in the path instead of at the end, and not closing the input files with a context manager like the output file, which suggests that this code is just pierced together from various sources.

A robust solution would require many more lines.

Re:Help us collect modern Python NumPy code solutions by hermitdev on Monday March 28, 2022 @12:54PM (#30832609)
For bonus points: how about properly handling UTF-8 byte order marks (BOM), too? The way this code is written if the files contain a BOM, you're going to pass the BOM right into the destination file. Fine for the first file, not so fine for any other file. Dropping a BOM mark in the middle of a file isn't going to result in valid UTF-8...
Re:Help us collect modern Python NumPy code solutions by mynameismon on Monday March 28, 2022 @10:31AM (#30830868)
Another memory issue (although unlikely) could also be the file itself being larger than the RAM size, which can be avoided by chunking the textfiles.