News for hackers. Stuff that matters.
Re:Show HN: Faster UTF-8 validator by pdkl95 on Saturday November 16, 2019 @12:58AM (#21551267)
[re: upfront parsing - a PSA about validating input]

> A lot of the software ... don't validate upfront

They almost always should be validating all input up front. Deferring validation tends to become many different pieces of code all informally parsing fragments of the input that are needed locally. Without upfront validation of the complete unit of input, the resulting fragmented parsers are just a weird machine waiting to be programmed by a malicious attacker.

For a much better explanation, I strongly recommend Meredith and Sergey's 28c3 talk[1] about The Science of Insecurity.

> you often want to tolerate invalid bytes and show a replacement character

While this validator wouldn't be useful in that situation, it is still important to validate the input upfront. When showing replacement characters, the "invalid bytes" that will be supported with a replacement character should be formally defined and added to the validation grammar, because they are no longer "invalid", but instead will be handled as a special case.

Postel's robustness principle shouldn't be used as an excuse to skip validation. Being "liberal in what you accept" should still be well-defined and validated.

[1] https://media.ccc.de/v/28c3-4763-en-the_science_of_insecurit...

Re:Show HN: Faster UTF-8 validator by nitwit005 on Tuesday November 19, 2019 @11:03PM (#21580990)
You trimmed my quote in a way that completly changes the meaning, so I'm not even going to bother responding to that.

How unicode replacement characters should work is already well defined. Just look it up.