News for hackers. Stuff that matters.
Re:First Impressions of Rust by jmgao on Thursday August 06, 2020 @09:08PM (#24077006)
If your String type must be valid Unicode (which is true for pretty much every language that isn't C++), it cannot represent all paths. On most unixy filesystems, filenames are arbitrary byte sequences which aren't valid UTF-8, and on windows, NTFS stores filenames as UTF-16, but it allows unpaired surrogates.
Re:First Impressions of Rust by throwaway894345 on Thursday August 06, 2020 @10:18PM (#24077368)
What languages require string types to be valid Unicode? Go for example let’s you put anything in a string; it’s only Unicode if you put Unicode into the string including via creating string literals.
Re:First Impressions of Rust by cptroot on Thursday August 06, 2020 @10:28PM (#24077412)
I think python also takes the approach where strings must be valid unicode. Both Rust and Python use their String types to represent valid unicode, and something else ([] and byte strings, respectively) to represent other encodings and invalid unicode.
Re:First Impressions of Rust by rkangel on Friday August 07, 2020 @06:13AM (#24080316)
Requiring strings to be valid unicode simplifies the internal workings of (and use of) a language greatly. Dealing with unknown input becomes a single operation at the boundary after which you can safely assume everything is validly printable or processable as text.
Re:First Impressions of Rust by throwaway894345 on Saturday August 08, 2020 @12:37PM (#24092960)
I think there is some convenience in being able to use the type system to denote input which has already been validated; however, I can't see how a language runtime benefits from this property with respect to unicode. Notable, printing and processing text aren't operations a runtime typically deals with. Libraries including the standard library might benefit from this property, but even then I wouldn't say the benefit is "great". It's a nice property, but I don't think it makes or breaks anything.
Re:First Impressions of Rust by rkangel on Saturday August 08, 2020 @01:51PM (#24093620)
It's worth pointing out that Python 3 thought it important enough to make a backwards incompatible change to separate 'bytes' from 'strings' to effectively get the same result.
Re:First Impressions of Rust by throwaway894345 on Saturday August 08, 2020 @03:21PM (#24094327)
Probably, but I think that’s more to do with the typing convenience (which is even more puzzling in Python’s case since it isn’t statically typed).
Re:First Impressions of Rust by ChrisSD on Thursday August 06, 2020 @10:25PM (#24077397)
Rust, Java, Swift...
Re:First Impressions of Rust by erik_seaberg on Thursday August 06, 2020 @10:03PM (#24077298)
Is that technically UCS-2 rather than UTF-16?
Re:First Impressions of Rust by ChrisSD on Thursday August 06, 2020 @10:11PM (#24077334)
Technically not since ~2000. Windows uses UTF-16 throughout the OS. That is, Windows strings will be encoded and decoded as UTF-16 not UCS-2.

However the tricky bit is that the kernel doesn't enforce this so it's possible for programmers to intentionally make broken UTF-16 strings. A broken UTF-16 string shouldn't be considered UCS-2 just because it happens to be a valid UCS-2 string (otherwise all bit patterns could be called "UCS-2" so long as they are an even number of bytes in length).