News for hackers. Stuff that matters.
Re:The string type is broken (2013) by brundolf on Tuesday June 22, 2021 @09:39PM (#27599766)
Rust's string type doesn't have the mentioned problems: https://play.rust-lang.org/?version=stable&mode=debug&editio...

Though I would bet this mainly has to do with being a language born in the age of the web. It seems like something that would be very hard to back-port (as opposed to designing for it from the beginning), but proper handling of Unicode is practically table-stakes for a new language in today's world. I would guess (I don't know) that Swift, Go, etc also do a good job with this stuff.

Re:The string type is broken (2013) by natanbc on Tuesday June 22, 2021 @10:00PM (#27599918)
It does for multi-codepoint characters (such as the rainbow flag): https://play.rust-lang.org/?version=stable&mode=debug&editio...

Elixir gets it right: https://replit.com/@natanbc/reverse

Re:The string type is broken (2013) by brundolf on Tuesday June 22, 2021 @10:03PM (#27599935)
Yeah, another commenter mentioned grapheme-clusters, though apparently there's a readily available crate that adds support: https://stackoverflow.com/questions/58770462/how-to-iterate-...

According to this they added it to the standard library at one point but then broke it back out because the lookup tables were too large

Re:The string type is broken (2013) by pjgalbraith on Tuesday June 22, 2021 @10:32PM (#27600131)
Re:The string type is broken (2013) by thurn on Tuesday June 22, 2021 @09:47PM (#27599822)
.chars() iterates over unicode scalar values, not actual grapheme clusters, so this code would also be incorrect in many circumstances. Ultimately it is incumbent on the developer to understand what their actual unicode-processing requirements are.
Re:The string type is broken (2013) by brundolf on Tuesday June 22, 2021 @09:49PM (#27599842)
I'm not familiar with what exactly falls into each category, though apparently there's an (official?) crate which extends the core str type to support those as well: https://stackoverflow.com/questions/58770462/how-to-iterate-...
Re:The string type is broken (2013) by qw3rty01 on Tuesday June 22, 2021 @10:10PM (#27599987)
this isn't quite set up the way the article was mentioning. Here's with it set up correctly:

https://play.rust-lang.org/?version=stable&mode=debug&editio...

It looks like it still misses some of the cases (specifically with decomposed vs precomposed).

As a side note, it looks like the playground editor also doesn't correctly render some of the lengths (the cursor is 1 character off on the decomposed case)

Re:The string type is broken (2013) by brundolf on Tuesday June 22, 2021 @10:14PM (#27600015)
Huh, I'd never heard of decomposed/precomposed before and I kind of skimmed over them in the original article. Looking at your example, I still can't figure out what the difference is between the two (I guess the Playground editor itself just can't represent the difference?). I'll have to go read up on that concept.

Edit: Having read about them, I guess I assumed that if my keyboard inputs an ë it's actually inputting those two underlying characters. Or to put it differently, I assumed there weren't two separate representations for the same character; that the "decomposed" and "precomposed" versions are one in the same, at the byte level. Learned something h̶o̶r̶r̶i̶f̶y̶i̶n̶g̶ new!

Re:The string type is broken (2013) by qw3rty01 on Wednesday June 23, 2021 @09:28AM (#27604319)
After a little bit of research, it looks like this is how you would get the expected outputs: https://play.rust-lang.org/?version=stable&mode=debug&editio...
Re:The string type is broken (2013) by unknown_error on Tuesday June 22, 2021 @09:44PM (#27599800)
What does "table stakes" mean? Never heard that before.
Re:The string type is broken (2013) by brundolf on Tuesday June 22, 2021 @09:45PM (#27599810)
"Baseline expectation". I think it comes from poker or something
Re:The string type is broken (2013) by bombcar on Tuesday June 22, 2021 @09:56PM (#27599893)
Table stakes are a required minimum bet in a game (often poker) - the meaning being if you can’t provide at least this you can’t play. So if a new language doesn’t support Unicode well it won’t get any adoption.
Re:The string type is broken (2013) by billforsternz on Tuesday June 22, 2021 @09:56PM (#27599886)
It's a gambling expression meaning the minimum [money] you need to even sit at the table. To be a really competitive player you need more.