News for hackers. Stuff that matters.
Re:How long would it take to create a Windows 1.0 capable ma... by bumblebritches5 on Wednesday May 30, 2018 @07:19PM (#17192733)
That's precisely what I meant with your UCS-2 comment.

What do you mean by variable width? the 4 bit header would be limited to the first code unit, the following ones would just be pure codepoint values.

and that's a good point, I as picturing it as the top 4 bits that describe the format, so the header's byte order would be field, but the actual codepoint value would be encoded with whatever byte and bit order the header indicated.

Re:How long would it take to create a Windows 1.0 capable ma... by Dylan16807 on Thursday May 31, 2018 @03:31AM (#17194612)
Oh, so it's a BOM? Why integrate it into the code unit if you're only going to have one of them?
Re:How long would it take to create a Windows 1.0 capable ma... by bumblebritches5 on Thursday May 31, 2018 @06:52AM (#17195369)
> Oh, so it's a BOM?

Kinda? It isn't limited to just the byte order tho, but I can see it.

> Why integrate it into the code unit if you're only going to have one of them?

There would be multiple code units, just 1 of the mini headers/BOM.

With that comment I was trying to contrast it to UTF-8 which has those 2 leading bits of the continuation code units set to `0b10`, it wouldn't have that.

Re:How long would it take to create a Windows 1.0 capable ma... by Dylan16807 on Thursday May 31, 2018 @10:35AM (#17196600)
Oh, the first code unit of each code point, not the first code unit of the text. That's how I had originally understood you until I got confused by the talk of byte order.

I have to say I really don't see the value of being byte-order flexible, but always requiring the header be in the "first" byte, because that makes it so big-endian and little-endian formats are packed completely differently. 2 and 4 byte code units will be misaligned all the time too, might as well just use a series of bytes in fixed order.

> What do you mean by variable width?

Make the first byte be like UTF-8 (so a prefix of 0, 10, 110, or 1110), and the following bytes be raw data. It compacts a lot better than a fixed-length header, with roughly the same level of complexity.

But now that I think about it a continuation bit is probably the best option here because it regains some ability to self-synchronize.