News for hackers. Stuff that matters.
Re:String types in Rust by kibwen on Tuesday March 29, 2016 @01:05AM (#11379587)
WTF-8 is great, and lest anyone think that it's intended to be a joke it's worth mentioning that it's actually a very useful thing:

"WTF-8 (Wobbly Transformation Format − 8-bit) is a superset of UTF-8 that encodes surrogate code points if they are not in a pair. It represents, in a way compatible with UTF-8, text from systems such as JavaScript and Windows that use UTF-16 internally but don’t enforce the well-formedness invariant that surrogates must be paired."

Though note that it's only suitable for internal usage (as in, you wouldn't expose an API that uses WTF-8), so it's not something that you need to actually think about when learning Rust. :P

Re:String types in Rust by derefr on Wednesday March 30, 2016 @06:39PM (#11393584)
I don't really get the point of it; I'd think that, properly, what you'd want is:

1. a plain "buffer of unparsed octets" type (like Erlang's "binary" type), to hold onto your weird invalid UTF-16 (or whatever other weird invalid stuff you like);

and 2. a strict, validated-on-construction "sequence of codepoints" type.

Going from #1 to #2 wouldn't be a direct cast; it'd instead be a stream-parser function that could be hooked with a callback, which would be handed parsed-and-valid, and invalid-and-left-unparsed, "segments":

   maybe_parsed_segment() = valid_string_of_codepoints() | binary()
...and would then respond with a valid segment, which could just be "" to discard the input segment.

The default callback would presumably respond to a binary() of size N by emitting a "�" string of length N. That's pretty much what we have today, but both sturdier and more extensible.