News for hackers. Stuff that matters.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by shawkinaw on Wednesday January 08, 2020 @10:04AM (#21992020)
This is using Punycode encoding, see https://en.m.wikipedia.org/wiki/Emoji_domain.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by jackewiehose on Wednesday January 08, 2020 @10:12AM (#21992083)
Why is this even a thing?
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by tialaramex on Wednesday January 08, 2020 @10:30AM (#21992227)
There are a bunch of different human writing systems. All of them are weird because they were invented by humans, most of them are very weird indeed because they were invented by humans a long time ago and then gradually mutated.

The Latin system is the one you're using here. It's very popular. Most humans in the integrated tribes are somewhat familiar with it‡. It has twenty six "letters" and then twenty six more "capital letters" which look different but mean almost the same thing for some reason, and then a bunch more symbols that aren't quite letters although some (apostrophe, ampersand) have a better claim than others. But other popular systems include Han, which has a shitload of logograms, and Cyrillic and Greek which have different letters than Latin and different rules about how letters work.

Anyway, the people who invented DNS only or primarily used the Latin system and they weren't much into capital letters. So, their system doesn't treat capital letters as different and only has one set of twenty six Latin letters, ten digits, a dash and an underscore for some reason.

But, lots of people who do NOT have Latin as the primary or only writing system found this annoying to work with. They wanted to use their writing system with DNS especially once the Web came along and popularized the Internet.

Punycode is a way to use some reserved nonsense-looking Latin text in any DNS label to mean that actually this DNS name should be displayed as some Unicode text. Unicode encodes all popular human writing systems (and lots of unpopular ones) fairly well, so this sorts the problem out. Specifically Punycode reserves Latin names that start xn-- for this purpose. By only caring about display this avoids changing anything in the technical underpinnings. Only user-facing code needed to change, every other layer is unaltered.

The rules about IDNs say that a registry (such as .com) should have rules to ensure the names in that registry are unambigous and meaningful. But in practice the purpose of the .com registry in particular is to make as much money as possible regardless of the consequences. So you can register any name you please and they won't stop you even if it's deliberately confusing.

‡ None of the extant unintegrated tribes have writing. This probably doesn't mean anything important.

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by downerending on Wednesday January 08, 2020 @12:36PM (#21993709)
Perhaps ironically, URLs were never meant for human consumption in the first place. You were meant to "travel" to various sites via search indices, etc. (Think Google.)

Viewed in that light, restricting DNS names to ASCII as a way to reduce bugs, security issues, etc., makes a lot of sense.

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by ncmncm on Wednesday January 08, 2020 @01:19PM (#21994180)
Search engines happened remarkably late. Nobody understood how easy search engines would be.

People thought you would follow links from directory pages linked from your "home page", hence the house symbol still seen in browsers. Yahoo! is a leftover of an attempt at a directory.

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by downerending on Wednesday January 08, 2020 @02:10PM (#21994669)
Indeed--this is what was meant by "indices".

It's kind of like XML. It was never meant to be seen by human eyes, except in a few debugging scenarios. Unfortunately, that intention was ignored, and now we have a usability disaster. (At least in the case of XML, it can just die.)

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by lokedhs on Wednesday January 08, 2020 @10:20PM (#21998256)
I agree with what you said about XML. But in its stead we have a lot of JSON which is several respects is even worse.

The worst part of it is that it doesn't have a stable definition for numbers, making it impossible to guarantee you're getting the same value back if you encode and then decode a number. Reliably preserving the data you serialise using it should be a primary feature for an encoding format. JSON can't even do that.

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by gowld on Thursday January 09, 2020 @03:51PM (#22005567)
Why do you need a stable serialization for an unstable data type? Use a string of you want stability.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by lokedhs on Thursday January 09, 2020 @09:07PM (#22007848)
The point is that a 64-bit integer is stable in the language I'm using (which is most languages).

My opinion is that a serialisation format that explicitly makes something as fundamental as the representation of numbers unspecified is not useful as a data representation format.

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by LadyCailin on Wednesday January 08, 2020 @02:23PM (#21994792)
Oh man, do you remember aol keywords?
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by downerending on Wednesday January 08, 2020 @02:51PM (#21995063)
And who can forget the original "me too"? :-)
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by WorldMaker on Wednesday January 08, 2020 @02:57PM (#21995111)
It's another reminder how much internet infrastructure predates the Unicode standard (along with other equally wacky hacks like email's UTF-7). URLs weren't restricted to ASCII versus Unicode for any sort of bug reduction/security issue/whatnot, they were restricted to (a subset of) ASCII simply because Unicode did not exist at the time to even be considered.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by gowld on Thursday January 09, 2020 @03:53PM (#22005604)
Unicode (1987-1991) predates URLs (1990-1994) but not DNS (1983)
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by WorldMaker on Thursday January 09, 2020 @06:18PM (#22006930)
The most critical Unicode date for this sort of discussion is the standardization of UTF-8 (1993), because HTTP (1989) piggy-backed on Telnet (1969), an 8-bit text channel.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by johnchristopher on Wednesday January 08, 2020 @01:38PM (#21994370)
> You were meant to "travel"

You were meant to surf the web ^^.

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by johannes1234321 on Wednesday January 08, 2020 @03:19PM (#21995331)
"Surfing" came later afair it's traced back to some Australian librarian who had a surf board on here mousepad while looking for an analogy for an article she wrote. [Citation needed]
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by johannes1234321 on Wednesday January 08, 2020 @03:17PM (#21995314)
I won't judge URLs, but we are talking about DNS and hostnames here, which predate URLs and we're ment to give human-readable names to hosts.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by gowld on Thursday January 09, 2020 @03:47PM (#22005516)
Citation needed that URLs were never meant to be seen. Every web browser had a location bar.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by billpg on Wednesday January 08, 2020 @12:24PM (#21993550)
I vaguely recall that the double-hyphen used to be disallowed in domain names, reserving them for future definition. "xn--" becoming that future definition.

I'm having trouble confirming this as the double-hyphen is now very much allowed, thanks to IDN.

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by anticensor on Wednesday January 08, 2020 @12:42PM (#21993772)
FYI, xn stands for eXtended Names.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by gowld on Thursday January 09, 2020 @03:45PM (#22005497)
A+ comment except for this part:

> so this sorts the problem out.

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by nisuni on Wednesday January 08, 2020 @10:37AM (#21992277)
That’s unnecessarily patronizing.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by nkrisc on Wednesday January 08, 2020 @10:43AM (#21992359)
I thought it was a well thought-out and clearly written answer to a question that comes across as dismissive.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by jackewiehose on Wednesday January 08, 2020 @11:08AM (#21992614)
Yes, I wrote it this dismissive because I already knew the answer and obviously don't agree with the whole idea but it's a good answer for people who don't know the relationship between emojis and character encodings.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by davidcollantes on Wednesday January 08, 2020 @11:03AM (#21992566)
Nothing patronising about it. I find it interesting, and educative.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by Avamander on Wednesday January 08, 2020 @10:27PM (#21998294)
Just like the comments that say "ASCII ought to be enough for everybody"?
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by iso1210 on Wednesday January 08, 2020 @11:06AM (#21992596)
Imagine if DNS consisted only of Arabic characters. Imagine how grumpy you'd be that you couldn't go to newyork.gov.us for your local government website, and instead had to go to نيويورك instead?

Assuming you can't read Arabic, how on earth would you even recognise that address, let alone remember how to type it.

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by jackewiehose on Wednesday January 08, 2020 @11:27AM (#21992831)
Well, I'd probably had to learn something like this: https://en.wikipedia.org/wiki/Arabic_chat_alphabet

I always assumed the non-latin-alphabet people know their way around this because they already used the internet before unicode-domains.

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by JadeNB on Wednesday January 08, 2020 @12:37PM (#21993715)
> I always assumed the non-latin-alphabet people know their way around this because they already used the internet before unicode-domains.

But that's not a very good reason to continue it (besides which, there are new people coming to the Internet every day).

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by anon31945710 on Thursday January 09, 2020 @05:00AM (#21999885)
That (Arabizi and other romanizations) are in wide use now specifically because the latin alphabet was the de facto standard in the computer world. In hypotheical above, that is not likely the case, and you would have to use actual Arabic script.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by wongarsu on Wednesday January 08, 2020 @12:03PM (#21993293)
In Japan there's a trend to use phone numbers as domain names, probably for this very reason.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by iso1210 on Tuesday January 14, 2020 @05:52AM (#22043616)
Even then those numbers are western. I guess it's easy enough to learn - I'm useless at languages but learnt to recognise arabic and urdu numbers. At least almost every culture in the world has a base 10 system for encoding numbers -- learning Gujarati numbers in addition to your native number system is trivial.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by jackewiehose on Wednesday January 08, 2020 @01:28PM (#21994268)
Interesting. I agree that latin isn't sufficient but they could have just extended the list of allowed characters by a few other defined alphabets. No need to include every(?) unicode-point just because it's the technical more elegant solution.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by wongarsu on Wednesday January 08, 2020 @01:56PM (#21994553)
Allowing only a few alphabets doesn't solve much (Cyrillic "а" still looks like latin "a" in most fonts) but opens new cans of worms: how do you update this list, why isn't alphabet x of important minority y included, etc.

Much easier to just allow everything in the technical standards and let domain registries set reasonable standards that make sense for that tld. Sometimes that works well (.de for example has a list of 93 allowed unicode characters [1] which covers everything a German might plausibly use from German or neighboring country's alphabets), but some registries just don't seem to care much (e.g. .com).

1: https://www.denic.de/en/know-how/idn-domains/idn-character-l...

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by wahern on Wednesday January 08, 2020 @03:52PM (#21995668)
DNS was already 8-bit clean. IDNA was chosen for a complex set of compatibility, management, and user interface reasons.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by blacksmith_tb on Wednesday January 08, 2020 @12:24PM (#21993556)
I have a few years of rusty undergrad Arabic fading in my past, and my take is that non-native speakers would struggle with an ad hoc transliteration system like this (imagine a non-native English-speaker trying to read 133tspeak...)
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by retrac on Wednesday January 08, 2020 @10:17AM (#21992120)
Punycode? It's to allow non-ASCII characters in domain names without breaking compatibility with all the standards and software that specify that domain names only use ASCII characters.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by alangpierce on Wednesday January 08, 2020 @11:12AM (#21992671)
Unicode and Punycode were originally designed for letting computers handle the wide variety of human writing systems, common ones like Chinese and Arabic, as well as rarer ones. Much more recently, the emoji characters were added to that same Unicode system, so any encoding system these days that's international-friendly also ends up supporting emoji as well.

So Punycode wasn't specifically designed to work with emoji, it just happens to work with emoji because emoji are part of Unicode.

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by fennecfoxen on Wednesday January 08, 2020 @10:34AM (#21992245)
It turns out sometimes people might just want to register mañana.com or 魔法少女まどかマギカ.jp or whatever.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by asveikau on Wednesday January 08, 2020 @10:18AM (#21992125)
In a lot of languages it is good to be able to render something other than us-ascii. In many places the Latin alphabet without diacritics will be well understood and no problem, but that isn't true everywhere.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by tsavola on Wednesday January 08, 2020 @10:15AM (#21992108)
Do you mean Punycode, or this joke?
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by DarkWiiPlayer on Wednesday January 08, 2020 @10:19AM (#21992140)
Punycode is the joke
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by wongarsu on Wednesday January 08, 2020 @11:04AM (#21992574)
Limiting dns names to 26 latin characters and the 10 arabic digits is the joke. From the top of my head I can't come up with a single language other than English that doesn't use additional letters (or at least additional modifiers on some letters). Punycode is the most sane solution to an insane situation.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by DarkWiiPlayer on Thursday January 09, 2020 @05:53AM (#22000119)
DNS should have been UTF-8 from the beginning; that would have been a good solution. As things stand, slapping punycode on it is a terrible solution to a dumb problem.

But then again, when you invent something, you decide how it works, and the world could just re-invent a more international version of DNS but chose not to.

Also, ASCII is still probably the best charset to use, if you have to choose one; it can represent most possible sounds in some way and symbols represent single sounds (as opposed to, for example, chinese characters). It's very widely used (as opposed to, for example, the greek alphabet).

So yes, limiting DNS names to 26 latin characters and 10 arabic digits was probably the best option at the time.

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by DarkWiiPlayer on Thursday January 09, 2020 @05:41AM (#22000068)
Even english has some borrowed expressions like "à la" that can are more correctly written with non-ascii characters, so not even english is really safe, if you want to be a bit fancy about it.
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by samatman on Wednesday January 08, 2020 @11:56PM (#21998681)
Swahili, Hawaiian, Italian, that's just off the top of my head, there are many others.

Hawaiian uses ' as a proper letter, granted.

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by Liquid_Fire on Thursday January 09, 2020 @10:35AM (#22001998)
Doesn't Italian use accented vowels in some cases?
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by samatman on Thursday January 09, 2020 @12:34PM (#22003372)
Ah, yes of course it does. Silly me.

None of this detracts from the greater point, being that we need a way for all writing systems to somehow squeeze down into the subset of ASCII supported by legacy protocols.

Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by PeterisP on Wednesday January 08, 2020 @02:05PM (#21994624)
Well, latin works, but it's not that popular outside of Vatican nowadays :)
Re:Add '127.0.0.1 xn–9q8h' to /etc/hosts gives you "localghost" by carapace on Wednesday January 08, 2020 @04:22PM (#21995940)
DNS is the joke, punycode is one of the punchlines. ;-)