News for hackers. Stuff that matters.
Re:UCS vs. UTF-8 as Internal String Encoding by weinzierl on Thursday May 28, 2015 @03:28PM (#9620541)

    >> What is the practicality of an unbounded number of   
    >>  possible graphemes?

    >That question doesn't really make sense. Unbounded  
    >cluster sizes is simply a result of unicode design 
    >because there's no reason to bound it.
The standard itself doesn't mention any bounds but there is Unicode Standard Annex #15 - Unicode Normalization Forms which defines the Stream-Safe Text Format.

    UAX15-D3. Stream-Safe Text Format: A Unicode string is said to be in             
    Stream-Safe Text Format if it would not contain any sequences of
    non-starters longer than 30 characters in length when normalized
    to NFKD.

    * Such a string can be normalized in buffered serialization with a
      buffer size of 32 characters, which would require no more than 
      128 bytes in any Unicode Encoding Form.
                                                                                     
    * Incorrect buffer handling can introduce subtle errors in the
      results. Any buffered implementation should be carefully checked
      against the normalization test data.
                                                                                     
    * The value of 30 is chosen to be significantly beyond what is
      required for any linguistic or technical usage. While it would have              
      been feasible to chose a smaller number, this value provides a very              
      wide margin, yet is well within the buffer size limits of practical              
      implementations.
                                                                                     
    * NFKD was chosen for the definition because it produces the
      potentially longest sequences of non-starters from the same text.
Funny thing I noticed when looking this up: The The Unicode Standard Version 7.0.0 PDF has exactly 1000 pages.

[1] http://www.unicode.org/reports/tr15/#Stream_Safe_Text_Format

There are no comments on this comment yet.