Memory use in CPython and MicroPython

Re:Memory use in CPython and MicroPython by ubernostrum on Monday July 03, 2017 @04:20AM (#14686733)

My use case is that I don't want iterating/indexing to yield the underlying byte representation, since that may end up slicing up a codepoint large enough to require multiple bytes.

I want the fundamental constituent unit of my strings to be the thing that the string is conceptually made of. When I'm ready to turn it into a sequence of bytes, I'll do that myself.

[ Parent | on HN ]

Re:Memory use in CPython and MicroPython by patrec on Monday July 03, 2017 @06:20PM (#14691888)

But the text is not conceptually made of unicode code points. It's conceptually made of characters. Codepoints are basically ever only used for either

a) low-level mucking around like normalization -- a fringe use

b) a (in the general case) bad approximation to characters -- the normal use, which will produce bad results for various "atypical" cases

Now, personally, I'd often rather muck around with bytes because frequently they are a more useful abstraction than codepoints and lots of in practice important text operations can be performed quite fine (and efficiently) on them, even if the text is full of non-ascii characters.

[ Parent | on HN ]