News for hackers. Stuff that matters.
Re:Announcing Unicode 7.0 by buster on Tuesday June 17, 2014 @08:03AM (#7903877)
Do we really need small pictures in fonts?
Re:Announcing Unicode 7.0 by gtirloni on Tuesday June 17, 2014 @09:15AM (#7904193)
I had the exact same question pop up in my mind after seeing the emoji's. I understand all characters can be considered nothing but pictures, but a thermometer, really? Is this used in any widespread language to be of significance to Unicode? Is Unicode trying to also offer some kind of clipart capability?

I think we've seen our fair share of derailed projects in IT. Unicode is certainly a very critical project, which makes its focus even more important.

Re:Announcing Unicode 7.0 by sho_hn on Tuesday June 17, 2014 @09:51AM (#7904381)
Unicode contains a wide range of different scripts.

For example, some are phonographic (the letters represent phonemes, i.e. speech sounds), others are logographic (the letters represent words, or morphemes).

Both of these can be featural: The Korean Hangul script is phonographic, but the letter shapes are not arbitrary. Some of them are visualizations of mouth shape or tongue position when producing the sound. For example the velar consonant ã„ą (somewhere inbetween the English g and k, roughly) is the tongue position as seen from the side.

A logogram can also be featural, i.e. it can look like the thing it represents. This would be the case for a glyph that means "thermometer" and looks to viewers like a thermometer. It could also be arbitrary - some shape that you just agree on to mean thermometer. Perhaps sharing some graphical aspects with topically related logograms.

They're all just spots on a spectrum.

(Edit: I briefly touched on pictograms vs. ideograms a bit more down below.)

Re:Announcing Unicode 7.0 by elfcard on Tuesday June 17, 2014 @09:57AM (#7904415)
That's amusing, because I looked at the thermometer, and thought, oh that's good, I can use that. But a chilli pepper, do we really need a character for a chilli pepper? So to answer your question, unicode is trying to bridge the gap in communication that offers a language independent set of recognizable glyphs that can be used to communicate. And one mans chilli pepper is another mans thermometer.
Re:Announcing Unicode 7.0 by yoz-y on Tuesday June 17, 2014 @11:26AM (#7904947)
I suppose that the chilli pepper is there for restaurant menus which often use it as an indicator of spiciness.
Re:Announcing Unicode 7.0 by greggman on Tuesday June 17, 2014 @10:25AM (#7904580)
Is the thermometer symbol used widely? No idea. Is emoji used widely? Yes, In Japan ðŸ‡ŊðŸ‡ĩ (and Korea? 🇰🇷) and has been for > 15 years. A common email/msg composed by a Japanese woman 👧 age 15 to 35 uses many emoji.

I realize this message will not render correctly in Chrome because Chrome, at least Desktop Chrome, does not yet support emoji 😭

Re:Announcing Unicode 7.0 by jaweegian on Tuesday June 17, 2014 @03:22PM (#7906414)
Is the problem with Chrome or the OS and font selection? I think everything renders on my Win 7 + FX 30 setup I'm using now. Generally, I have the best success with Win 8 + IE 11.
Re:Announcing Unicode 7.0 by dubya on Tuesday June 17, 2014 @10:12PM (#7908209)
On OS X, Chrome renders boxes, Safari renders the proper characters.
Re:Announcing Unicode 7.0 by smackfu on Tuesday June 17, 2014 @10:12AM (#7904498)
That's not really the right question. We already have small pictures in fonts, just look at Wingdings or Webdings. Unicode is just assigning them to unique code points so they can be represented in a document without requiring the font to be specified.
Re:Announcing Unicode 7.0 by buster on Tuesday June 17, 2014 @10:21AM (#7904565)
And you consider Wingdings a good thing in the first place?! It's an abomination that should never have happened.
Re:Announcing Unicode 7.0 by gioele on Tuesday June 17, 2014 @10:36AM (#7904641)
But it did happen. And it has been used a lot.

So, for the sake of interoperability and the ability to read back these documents in a future Unicode-only world, we have to encode them.

Re:Announcing Unicode 7.0 by smackfu on Tuesday June 17, 2014 @10:36AM (#7904646)
Never said that, but the current situation is that we have glyphs that people are using in documents that can't be represented in Unicode, and that's pretty much the main reason there are Unicode revisions in the first place.
Re:Announcing Unicode 7.0 by buster on Tuesday June 17, 2014 @03:31PM (#7906469)
Why would documents using Webdings stop working today? I'm pretty sure they still work. That's a lame excuse. Considering your point, wouldn't it have been the chance to clean up this mess with unicode instead of piling up shitty icons in a standard?
Re:Announcing Unicode 7.0 by greggman on Tuesday June 17, 2014 @10:36AM (#7904649)
Yes, to support 15 years of Japanese communication since they were introduced to cellphones in Japan in 1998 and the majority of users here use them.
Re:Announcing Unicode 7.0 by sisk on Tuesday June 17, 2014 @01:42PM (#7905781)
You might not need them but consider this for a moment:

When I was a kid, there was a boy in my class who was mute. He carried a chalkboard and flashcards to facilitate communication. Eventually, he was chosen to pilot a device that was sort of like a keyboard with oversized keys. Each key had a pictograph on it[0].

Wouldn't it be nice for him if the flashcards and keyboard device had a consistent set of characters that were easily recognizable not only by him but by the people with whom he was communicating?

Now, would he ever need "MAN IN SUIT LEVITATING"? Probably not (you never know!) but I'm glad there is a standard set of otherwise useful pictographs for people like him.

[0] - I'm in my late 20s but this was when few families owned computers, let alone super-helpful handheld devices. Also worth mentioning that this boy knew ASL but attended a regular public school where few students and faculty could so the chalkboard and flashcards were infinitely more practical.

Re:Announcing Unicode 7.0 by buster on Tuesday June 17, 2014 @03:38PM (#7906495)
And that device would have a keyboard with thousands of keys?

Btw. that device exists today, and it's called a tablet. I'm pretty sure i have seen stuff exactly like that and it serves a great purpose.

The consistent set of "characters" (pictures in this case) would still look very different on each device or application that uses another font. Because a banana in one font could look very different in another font. The only thing that actually would be consistent if the content he uses would use actual pictures! (A website displaying a banana on his touchpad can look different on his laptop, can't it?).

Re:Announcing Unicode 7.0 by rational-future on Tuesday June 17, 2014 @08:21AM (#7903935)
That is what fonts are - sets of small pictures, accessible by an index.
Re:Announcing Unicode 7.0 by buster on Tuesday June 17, 2014 @08:26AM (#7903950)
I really don't agree on your definition of fonts. When i write a letter i am certainly not painting rows of little pictures on a sheet of paper.
Re:Announcing Unicode 7.0 by justincormack on Tuesday June 17, 2014 @08:40AM (#7904015)
You are if you write hieroglyphics. From another point of view you are when you write English too.
Re:Announcing Unicode 7.0 by buster on Tuesday June 17, 2014 @08:53AM (#7904072)
So, putting poo into unicode is just the same as hieroglyphs? Or hebrew or arabic or chinese characters? Still don't agree.
Re:Announcing Unicode 7.0 by sho_hn on Tuesday June 17, 2014 @09:39AM (#7904318)
How are emoji different from other logograms?
Re:Announcing Unicode 7.0 by rejschaap on Tuesday June 17, 2014 @10:12AM (#7904496)
There is a good case for adding ancient languages to Unicode. I can also see the case for adding smiley faces, which could be seen as a contemporary global language. But I fail to see the case for pictures of poop, pizza and cowbell. That just seems a bit random to me. Maybe everyone is drawing the line somewhere else, but I'm guessing most people don't want to go down the road of adding company logo's for Coca-Cola and McDonalds or pictures of the Americans presidents or the sigils from Game of Thrones. It's good that Unicode is extendable and all, but that doesn't mean we should be adding glyphs just for the sake of it.

Also, adding political glyphs is a sure way to ruin the universality of Unicode (I'm looking at you no-piracy).

All that said, I love Unicode, it has flaws but I think it's brilliant (no sarcasm).

Also, can't wait to actually use the man-in-business-suit-levitating-glyph (sarcasm).

I'd use the Unicode sarcasm marks, but your font probably doesn't support them yet...

Re:Announcing Unicode 7.0 by scrollaway on Tuesday June 17, 2014 @07:49PM (#7907754)
> Also, adding political glyphs is a sure way to ruin the universality of Unicode (I'm looking at you no-piracy).

As someone else pointed out, it comes from Wingdings, for which there has been an effort to fully get into Unicode.

https://en.wikipedia.org/wiki/Webdings#mediaviewer/File:Webd...

Re:Announcing Unicode 7.0 by meepmorp on Tuesday June 17, 2014 @09:54AM (#7904392)
I don't think emoji are logograms. Logograms represent actual words in a language, as opposed to emoji which don't have a conventional mapping to words (unless you consider the unicode character names such mappings, which is odd since most users don't have any idea what the character names are).
Re:Announcing Unicode 7.0 by sho_hn on Tuesday June 17, 2014 @10:07AM (#7904462)
You're right, but at the same time the level of abstraction it offers over spoken language is one of the reasons for the historic success of logographic writing. For example, in Chinese history it allowed mutually unintelligible dialects to more easily exchange information in writing, and it isolated orthography from shifts in pronunciation to a greater degree than possible with phonographic scripts. I feel like emoji have some similar properties along these lines, but of course also their own set of problems (specific visualizations can become dated, for example).

A fair bunch of Chinese characters are also actually pictographic (they're an image of something) or ideographic (they represent an abstract idea, not a morpheme or word) in origin, if not current usage.

This stuff - synergies and conflicts between writing systems and various media - fascinates me. Chinese writing for example now suffers from the problem that if your medium doesn't allow free-form painting/graphical compositing (like our computers largely don't right now) you can't easily coin new logograms, since you need a central registry that's slow to distribute to leaf nodes (cf. Unicode). So what's happening now is that new words get coined by recombining existing characters based on their sound value, adding a phonographic layer on top. And of course inputting Chinese has also become dependent on auxiliary systems like pinyin-based IMEs.

Meanwhile, the Korean Hangul alphabet has the interesting property that some of the letters graphically derive from each other with consistent patterns of "take this, add a stroke and you get this", making it easier to design keyboards with reduced numbers of buttons to enter combos. There's the theory that this contributed to the success of texting/mobile in Korea, and is a contributing reason for the healthy mobile industry there.

I'm getting far from the original topic with this rambling, but it still reads on it in one sense: There's a lot of variety to writing systems, and it's worth thinking about emoji as a spot on the spectrum instead of in isolation.

Re:Announcing Unicode 7.0 by meepmorp on Tuesday June 17, 2014 @12:40PM (#7905413)
> I'm getting far from the original topic with this rambling, but it still reads on it in one sense: There's a lot of variety to writing systems, and it's worth thinking about emoji as a spot on the spectrum instead of in isolation.

Except that emoji aren't a writing system. Writing systems encode language. Emoji encode emotions, perhaps, or suggest collections of ideas/emotions, but don't convey language per se because there's no conventionally defined correspondence between words in a language and a given emoji.

If I give you a string in English or Chinese or Arabic or Amharic, using the conventional writing systems for those languages, each string will be interoperable as a specific sequence of words in that language. A string of emoji doesn't work that way.

Re:Announcing Unicode 7.0 by sho_hn on Tuesday June 17, 2014 @12:52PM (#7905486)
Are you suggesting we turn falling into a specific set of relationships between symbols an entry requirement for the Unicode database, though? How would that look like in practice? I agree you're highlighting a useful difference, but I'm not sure about implications.
Re:Announcing Unicode 7.0 by meepmorp on Tuesday June 17, 2014 @02:36PM (#7906110)
Nah, I think keeping unicode as "characters" is fine. It's perfectly reasonable to encode non-linguistic data as characters if they're using in such wise.

Edit: to circle back, you said:

  How are emoji different from other logograms?
and everything thereafter was pointless pedantry on my part. Sorry for wasting valuable minutes of your life.
Re:Announcing Unicode 7.0 by sho_hn on Tuesday June 17, 2014 @02:59PM (#7906275)
No waste at all, you raised a good point :). Emoji are technically ideograms, not logograms ... though some of the Han characters are ideographic in origin/conception too (and then you get into cool things like compound ideograms), they just didn't stay purely ideographic.
Re:Announcing Unicode 7.0 by buster on Tuesday June 17, 2014 @09:59AM (#7904426)
I don't even understand your question. How is a character of a normal language (and i consider chinese normal) different from the picture of crap? And why would we put up a huge library of tiny pictures? If i want to look at cat pics, i don't open up the font browser. And where is the HN-logo unicode point? Can i register my own somewhere?

Unicode should be used to represent language and not anti-piracy symbols or poo or snowmen. Just my opinion, though. Obviously a great many people like shit in their fonts.

Re:Announcing Unicode 7.0 by sho_hn on Tuesday June 17, 2014 @10:17AM (#7904533)
So what about all the characters in Unicode that map to different phonemes or words in different languages? For example, Latin letters are sometimes used for different phonemes in different languages (and dialects, and applications like romanization), and Han logograms are sometimes used for different words/morphemes in different languages. How is that different from "picture of crap" mapping to different specific words/phrases in different languages? You can certainly point at a picture of a pile of crap and name it in many normal languages.

But wait - what's a "normal language" and what's a normal graphical representation of it? Is it down to the development process? A lot of writing systems were originally designed/agreed upon by small groups of people, too, for example.

My questions (and other comments in this subthread) aim to get you to think more abstractly about the problem space and define your beef more accurately, because I'm interested in this question (where should Unicode draw the line) as well. Let's brainstorm, basically.

Re:Announcing Unicode 7.0 by buster on Tuesday June 17, 2014 @10:46AM (#7904703)
The distinction between normal language and graphical representation is the same as what is clipart and what is a official part of a language. Is poo a letter? No. Is A a letter? Yes.

Fonts are not a clipart gallery and not a place for some committee to dump little pictures. If you want to show me a snowmen over the internet, use an image! Certainly people can understand an image of a snowmen.

Re:Announcing Unicode 7.0 by sho_hn on Tuesday June 17, 2014 @10:55AM (#7904763)
But įēŠ is also a character and means poo as well. It might not look like poo to you, but there's also examples of pictograms in the same script: į”° means field and looks like one.

Of course these are "official part of a language". But why? Under whose authority? You could cite "actual usage", but many of the emoji in Unicode certainly have seen actual usage for 10+ years in East Asia, so they qualify. And many of the letters in Unicode were in fact designed by committee prior to mass adoption by an actual population (for example the Hangul alphabet), so a lot of scripts already in Unicode originally didn't meet that barrier.

* = Letters vs. characters: Elements of alphabets are referred to as letters. Alphabets are graphical representations of phonemes, but many other writing systems encode syllables (syllabaries, like the Japanese kana), morphemes or words (logographies) instead. Unicode contains examples of all of these, and thus not just letters. So we want to talk about characters instead.

Re:Announcing Unicode 7.0 by buster on Tuesday June 17, 2014 @11:24AM (#7904931)
Oh, the irony: http://oi60.tinypic.com/21o111t.jpg

That's pretty much my point.

Re:Announcing Unicode 7.0 by sho_hn on Tuesday June 17, 2014 @11:32AM (#7904981)
For the record, the example was a Chinese character, which qualifies as "normal" according to the rulebook you set up. It's in active use in Mandarin and Japanese, and I'd think recognizable to many others (e.g. Cantonese readers or Koreans and Vietnamese). And it'll certainly show just fine on the default installs of most systems on the market right now.

http://www.unicode.org/cgi-bin/GetUnihanData.pl?codepoint=7C...

You're just being petty or culturally arrogant at this point, TBH.

Re:Announcing Unicode 7.0 by buster on Tuesday June 17, 2014 @03:28PM (#7906450)
The point is that it will never happen that all codepoints are available everywhere (every font, evers OS, every device), and i certainly don't want a gazillion of pictures loaded on my smartphones little memory, just because some people think it necessary to have a symbol for poop in fonts which is never used anyway. I even consider it a total waste in my laptops 16GB of memory, to be honest. It's not a waste when it serves the real purpose of a language. A chinese word for poop? Sure. A picture of actual poop? No. And where does it stop? Can i have some dog poop and cat poop codepoints please? I'm sure elephant poop looks very different to cat poop, i want a code point for that too.

I say it again: If you want pictures and cliparts, use .png. If you want to write actual text, use fonts. It's as simple as that.

Re:Announcing Unicode 7.0 by scrollaway on Tuesday June 17, 2014 @07:39PM (#7907713)
If you think using pngs instead of fonts is going to save your oh-so-precious mobile device's memory, you got another thing coming.
Re:Announcing Unicode 7.0 by buster on Wednesday June 18, 2014 @04:06AM (#7908989)
Which only shows that you know nothing. A unicode font can come in at about 20MB, but let's add more tiny pictures and make it 100MB, so that i can see the snowman on your webpage (which would be probably 2-5 kB as png).

Why should my browser load thousands of pictures that are never used?!

Yes, it's ridiculous.

Re:Announcing Unicode 7.0 by scrollaway on Wednesday June 18, 2014 @05:47PM (#7913076)
> A unicode font can come in at about 20MB, but let's add more tiny pictures and make it 100MB

You're pulling numbers out of your ass. You also clearly don't know how font fallbacks work, despite having been told how they work in this very thread. Several times.

Re:Announcing Unicode 7.0 by buster on Thursday June 19, 2014 @01:51AM (#7914653)
No, Arial Unicode MS: 22,1 MB. https://en.wikipedia.org/wiki/Unicode_font

And you clearly don't understand that a) this thread contains unicode that my browser still doesn't show, despite your magic fallback and b) the fonts stil contain garbage that has nothing to do with fonts.

Re:Announcing Unicode 7.0 by acdha on Tuesday June 17, 2014 @02:28PM (#7906068)
So you're using Windows XP or something similarly mired in the 90s which doesn't have fonts installed with glyphs for a standard character. That's bad for you and a small number of other people but that hardly says it's not useful to the billion or so people who can read Chinese.

The nice part about having it Unicode is that if you copy and paste that character somewhere else, it'll still work. It never goes through what we used to have to deal where the same character value couldn't reliably be processed without knowing the intended encoding with certainty.

Re:Announcing Unicode 7.0 by buster on Tuesday June 17, 2014 @03:22PM (#7906413)
No, Chrome, Debian testing.
Re:Announcing Unicode 7.0 by TorKlingberg on Tuesday June 17, 2014 @10:27AM (#7904592)
They are in Unicode for compatibility with other encodings, to make it easier to switch to Unicode. Most of the emoji comes from Japanese mobile phone carriers or fonts like Wingdings.
Re:Announcing Unicode 7.0 by stephen_g on Tuesday June 17, 2014 @08:25AM (#7903945)
Why not? It's not like every font of every typeface has to implement every Unicode code point.

You'd probably only have one or two installed that have the emoji characters.

Re:Announcing Unicode 7.0 by buster on Tuesday June 17, 2014 @08:30AM (#7903969)
Because it's waste. Because if not every font implements it, what is the usefulness of it? Did you ever visit websites that use glyphs your font doesn't know? How would you use those pictures on a website, when you can't even guarantee the style of the picture, because it may be implemented differently or not at all.

It just looks like a big waste for no good reason.

Re:Announcing Unicode 7.0 by masklinn on Tuesday June 17, 2014 @09:36AM (#7904309)
> Because it's waste.

Waste of what?

> Because if not every font implements it, what is the usefulness of it?

Most fonts only implements a fraction of unicode, is unicode a waste? Is cyrillic in unicode a waste? Is Cherokee? Braille symbols? Buginese and Meroitic cursive?

> Did you ever visit websites that use glyphs your font doesn't know?

Yes? The software will fall back on a font which does provide the glyphs, or display a block with the codepoint if it doesn't have any.

> How would you use those pictures on a website, when you can't even guarantee the style of the picture, because it may be implemented differently or not at all.

How would you use letters on a website, when you can't even guarantee the style of the letters, because it may be implemented differently or not at all?

Not only that, but unicode is not specific to websites for fuck's sake, not only will somebody find a use for them on websites even if you can't, somebody will definitely find a use for them outside of websites. Most (or all?) of the new emoji are inherited from windings[0] and webdings[1], they don't come out of nowhere.

> It just looks like a big waste for no good reason.

Again, a big waste of what?

[0] http://en.wikipedia.org/wiki/Wingdings

[1] http://en.wikipedia.org/wiki/Webdings

Re:Announcing Unicode 7.0 by mbq on Tuesday June 17, 2014 @10:30AM (#7904611)
For instance a waste of time of font designers; you can make rational decision about including certain alphabets (by analysing the target group demands), but selecting a reasonable subset of bazyllion random, singular pictures to draw and test is a major headache. Obviously this way the users get inconsistent experience.
Re:Announcing Unicode 7.0 by masklinn on Tuesday June 17, 2014 @10:53AM (#7904748)
Re:Announcing Unicode 7.0 by sho_hn on Tuesday June 17, 2014 @10:30AM (#7904610)
> Did you ever visit websites that use glyphs your font doesn't know?

Other comments have touched on this already, but I want to substantiate with additional information: In practice, fonts that cover many different writing systems at once are relatively rare. Your typical Western font will focus on Latin with varying coverage of descendants (like the Portuguese and Vietnamese alphabets) and perhaps expand to cover Cyrillic, but not more. Conversely, it's rare for a font by a Korean foundry to have equivalent Latin/Cyrillic coverage next to its native Hangul alphabet.

There's plenty of documents that mix these writing systems, though. I do in fact visit websites that aren't covered by any one font in my browser's default settings or any one family named in the site's markup/CSS all the time, and you probably do too.

Hence the font stacks on all relevant platforms perform glyph substitution: If font X doesn't contain a glyph, it'll go looking for a font Y that does and drop that glyph in. Many platforms allow you to manipulate substitution preference e.g. by defining aliases for font families.

Re:Announcing Unicode 7.0 by justincormack on Tuesday June 17, 2014 @08:39AM (#7904011)
Glyphs that are not supported should fallback to a font that does support them.
Re:Announcing Unicode 7.0 by JetSpiegel on Tuesday June 17, 2014 @09:51AM (#7904379)
Hopefully it will stop the nonsense of having custom fonts in websites for icons. Which can be copied and turn out to be a "t" or something.
Re:Announcing Unicode 7.0 by scrollaway on Tuesday June 17, 2014 @07:43PM (#7907732)
It won't. Those custom fonts you speak about not only use characters which aren't, and never would be in the unicode database (such as trademarked logos), but also draw from the private unicode blocks: https://en.wikipedia.org/wiki/Private_Use_Areas

Which means there's no sanely written font that would use a "t" for an icon. Wingdings is not a sanely written font.