Re:UTF-8 is a brilliant design
by nostrademons on Friday September 12, 2025 @07:56PM
(#45228072)
Sure, but for something like that whatever constructs the view can use an opaque index type like Animats suggested, which under the hood is probably a byte index. The slice itself is kinda the opaque index, and then it can just have privileged access to some kind of unsafe_byteIndex accessor.
There are a variety of reasons why unsafe byte indexing is needed anyway (zero-copy?), it just shouldn’t be the default tool that application programmers reach for.
You don't particularly want indexing for that, but cursors. A byte offset (wrapped in an opaque type) is sufficient for that need.
You could add a LUT for decently fast indexing as well. I believe Java does that.
Re:UTF-8 is a brilliant design
by naniwaduni on Friday September 12, 2025 @09:34PM
(#45228569)
You really just very rarely want codepoint indexing. A byte index is totally fine for view slices.
Re:UTF-8 is a brilliant design
by MrBuddyCasino on Saturday September 13, 2025 @03:14AM
(#45230014)
If you have multi-MB strings in an editor, that’s the problem right there. People use ropes instead of strings for a reason.
I agree though that usually you only need iteration, but string APIs need to change to return some kind of token that encapsulates both logical and physical index. And you probably want to be able to compute with those - subtract to get length and so on.