News for hackers. Stuff that matters.
Re:JEP254: proposal to represent Java Strings as ISO-8859-1 by MrBuddyCasino on Friday May 15, 2015 @06:17AM (#9549738)
I have always wondered why they didn't do that, it seems to be such a simple optimization. With UTF-8 you would give up some constant time operations like character lookups, while ISO-8859-1 shouldn't have any performance regressions I can think of. Because once you add a non-latin character to a String and the encoding has to change, the data has to be copied anyway due to String immutability.
Re:JEP254: proposal to represent Java Strings as ISO-8859-1 by pilif on Friday May 15, 2015 @08:37AM (#9550155)
>With UTF-8 you would give up some constant time operations like character lookups

if Java had a non-broken string API, even with the current UCS-2 based solution they couldn't do constant-time character lookups.

And even if Java went the extra mile of using characters big enough to encode every code point of a string, constant lookup would still not be possible because users are often interested graphemes, not characters.

Re:JEP254: proposal to represent Java Strings as ISO-8859-1 by MrBuddyCasino on Friday May 15, 2015 @10:17AM (#9550709)
That is sadly true, but at least for the Basic Multilingual Plane, charAt() and codePointAt() return correct results in constant time. The only non-broken methods are afaik codePointCount() and offsetByCodePoint(), which, as you pointed out, are not that interesting anyway.

Do you know of an example of a well-designed String API? I'd be interested in what trade-offs other language designers made (newer ones, like Rust/Swift/Go/?).

Re:JEP254: proposal to represent Java Strings as ISO-8859-1 by ygra on Friday May 15, 2015 @11:53AM (#9551399)
IMHO Python gets strings very right. The internal representation is variable, depending on the string's content (to save memory), and the external API only gives you access to code points.
Re:JEP254: proposal to represent Java Strings as ISO-8859-1 by TheLoneWolfling on Friday May 15, 2015 @01:17PM (#9552050)
The problem with Python strings is that innocuous operations can occasionally cause drastic memory usage ballooning.

A simple string append can, for instance, require up to 5x the memory consumed by the string.

Worse, this problem will only show up if someone ever enters in a high enough code point. Better hope any user-defined data never gets turned into part of a large string at any point, or you have a potential memory problem down the line.

Re:JEP254: proposal to represent Java Strings as ISO-8859-1 by gsnedders on Friday May 15, 2015 @12:02PM (#9551460)
That's only Python 3.3 and above; before that it was an under-defined mess, in CPython depending on compile-time options and often broken with the same UCS-2/UTF-16 mess as numerous other languages.
Re:JEP254: proposal to represent Java Strings as ISO-8859-1 by chrisseaton on Friday May 15, 2015 @06:22AM (#9549754)
I guess at the time the JVM was less aggressively optimised, and UCS-2 was a single simple approach that represented all required code points. Anything else would probably have looked like a micro-optimisation.
Re:JEP254: proposal to represent Java Strings as ISO-8859-1 by desdiv on Friday May 15, 2015 @10:07AM (#9550634)
At the time UCS-2 was the best thing they could choose from. Remember, Java predates UTF-8.
Re:JEP254: proposal to represent Java Strings as ISO-8859-1 by MrBuddyCasino on Friday May 15, 2015 @08:05AM (#9550040)
But back then we had less memory, and Java was never that memory efficient to begin with. I wouldn't say double-digit memory savings are in the realm of overly pedantic micro optimisations.

One language whose name escapes me took an approach where the internal representation is always unicode codepoints, and Strings have methods for projecting those .toUTF8() or whatever to interact with the outside world - I liked that a lot due to its simplicity, anyone knows which language that was?

Re:JEP254: proposal to represent Java Strings as ISO-8859-1 by agumonkey on Friday May 15, 2015 @06:55AM (#9549833)
Also fit well with World Wide Web + Write Once Run Anywhere : any OS in any locale.