Don’t try to sanitize input – escape output

Re:Don’t try to sanitize input – escape output by the8472 on Thursday February 27, 2020 @08:08AM (#22432868)

Most input by humans is unicode. Most people do not understand unicode. Don't try to sanitize it other than checking it's valid utf8 - which hopefully your programming language's string type/http parameter deserializer/DB engine already do for you.

For example some people think that stripping ZWNJ, ZWJ or other kinds of spaces is a thing they ought to do because it confuses their markup parsers or can be used to encode hidden information in posts or stuff like that. Guess what, it breaks emoji, arabic, some asian languages and a bunch of other things.

If you only sanitize your output and realize you made a mistake you can easily fix it by changing your output sanitzing algorithm. If you santizied your input you threw away data and can't fix your problem.

[ Parent | on HN ]

Re:Don’t try to sanitize input – escape output by nine_k on Thursday February 27, 2020 @09:55AM (#22433754)

See also: wtf-8 encoding, used by Mozilla for potentially broken Unicode input.

[ Parent | on HN ]

Re:Don’t try to sanitize input – escape output by klodolph on Thursday February 27, 2020 @10:34AM (#22434124)

WTF-8 is for a very specific application—to encode in an 8-bit sequence a 16-bit sequence which is nominally UTF-16 but may contain unpaired surrogates. It’s not used for weird user inputs, instead, it’s used for e.g. interop with the Win32 API.

[ Parent | on HN ]