Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on How UTF-8 Works. Parent comment
For example, this Chinese Wikipedia page is almost twice as big in UTF-16 as in UTF-8, because the vast majority of the content seems to be HTML tags:
$ curl -s 'https://zh.wikipedia.org/wiki/中华人民共和国' | wc -c 1877366 $ curl -s 'https://zh.wikipedia.org/wiki/中华人民共和国' | iconv -f utf-8 -t utf-16 | wc -c 3345724
$ curl -s 'https://zh.wikipedia.org/wiki/中华人民共和国' | tr -d '[:print:]' | wc -c 311226 $ curl -s 'https://zh.wikipedia.org/wiki/中华人民共和国' | tr -d '[:print:]' | iconv -f utf-8 -t utf-16 | wc -c 213444
$ curl -s 'https://zh.wikipedia.org/wiki/中华人民共和国' | tr -d '[:print:]' | gzip -9c | wc -c 98373 $ curl -s 'https://zh.wikipedia.org/wiki/中华人民共和国' | tr -d '[:print:]' | iconv -f utf-8 -t utf-16 | gzip -9c | wc -c 89031
There are no comments on this comment yet.
For example, this Chinese Wikipedia page is almost twice as big in UTF-16 as in UTF-8, because the vast majority of the content seems to be HTML tags:
If I strip out all the printable ASCII characters, it does indeed become about 33% more compact in UTF-16: With gzip compression it is only about 10% more compact: So I guess if you're archiving pure CJK text, maybe you could get a 10% benefit, though I suspect non-Unicode encodings of that text would be more compact anyway.