News for hackers. Stuff that matters.
Re:Memory use in CPython and MicroPython by dom0 on Sunday July 02, 2017 @11:43AM (#14682993)
Surrogate escaping has been part of Python since 3.2 or so which pretty much solved this.

Python's behaviour is only problematic when all of the below is true:

- Running on Linux and,

- using glibc and,

- glibc's locale files were deleted or the locale is otherwise not configured and,

- you try to read/write non-ASCII file names. (In this scenario Python falls back to the only locale it can process by itself, which is 7-bit ASCII -- the problem is not ASCII, rather the 7-bit restriction. I believe there is a bug open that suggests changing this fallback to UTF-8, which would solve this problem.).

The only time I saw that was with rear which did all of the above on purpose to save like 5 MB on the generated images. In all other cases there is no problem with any kind of file name.

Note that this is a severely broken system configuration where a lot of other stuff falls apart as well.

Re:Memory use in CPython and MicroPython by dom0 on Sunday July 02, 2017 @11:59AM (#14683057)
(Generally speaking I would greatly prefer it if CPython wasn't so reliant on the libc. For example, on a 32-bit system you'll typically find that CPython suffers from y2036 for many date and time operations, including formatting. This would of course add a lot of size to the already big CPython distribution.)