Ftfy – fix Unicode that's broken in various ways

Re:Ftfy – fix Unicode that's broken in various ways by zlynx on Tuesday January 09, 2018 @12:42PM (#16108017)

If the URL is "on the wire" it had better be 7-bit ASCII. Actually even more restrictive than that. Because that's the spec. https://tools.ietf.org/html/rfc3986

In user interaction with a browser or wherever else it seems that anything goes.

[ Parent | on HN ]

Re:Ftfy – fix Unicode that's broken in various ways by rspeer on Tuesday January 09, 2018 @01:03PM (#16108203)

That's the spec but this is also the spec: https://www.ietf.org/rfc/rfc3987.txt

It's true that protocols such as HTTP only use ASCII URIs on the wire. If you are implementing an HTTP client yourself, you will need to implement percent-encoding.

Which is different from saying "well, I know URLs should be ASCII on the wire, so I can safely delete all the non-ASCII characters from a URL." That's not true.

[ Parent | on HN ]

Re:Ftfy – fix Unicode that's broken in various ways by Piskvorrr on Wednesday January 10, 2018 @03:11AM (#16113271)

As soon as you add or remove to the string, it no longer points to the same resource; I considered this too obvious to mention. OTOH, for _validation_, this is useful: "you have a ZWJ character in an URL, that's unlikely". And yes, I understand that there are protocols that allow you to pass around the full Unicode or aunt Matilda or whatever - I should have been more specific.

[ Parent | on HN ]