News for hackers. Stuff that matters.
Re:Golang's network stack attempts to parse HTTP headers as ... by pansa2 on Thursday January 21, 2021 @08:21AM (#25858518)
> Python 3000 effort itself as wanting to insist "everything is Unicode"

How does Python 3 handle these HTTP headers and other things that are purely ASCII? Does it treat them as type `bytes` instead of (Unicode) `str`?

Re:Golang's network stack attempts to parse HTTP headers as ... by saurik on Thursday January 21, 2021 @08:36AM (#25858629)
I just did some digging, and the Python 3 built-in HTTP stuff seems to represent HTTP headers using a reasonably-comprehensive mechanism that is shared with MIME parsing and the various IMF semantics, and so I am pretty sure you end up getting back an object of type email.header.Header (but I might have misunderstood the documentation just now... I mostly gave up on Python when they moved to Python 3 so I haven't experienced this first hand).

https://docs.python.org/3/library/email.header.html#email.he...

This actually looks like a really solid implementation--like, I am super impressed by this (though watch it suck... everything sucks ;P)--as it doesn't assume any ability to flatten the result! That said, I am also deeply concerned that this might pre-decode something that should actually be in ASCII all the way down from MIME (which, notably, is complex enough of a format to include support for embedded comments). I kind of want to do some auditing here, to make sure there aren't some parser discrepancy vulnerabilities in common Python HTTP frameworks ;P.

But like, I think this object represents a sequence of strings each with its own encoding, allowing it to "Q" encode the result exactly as specified (and correctly round trip something it parsed, which still does feel a tad dangerous, and maybe the API lets you avoid it in those cases). It then has an __str__ implementation (to let you use it as a string) that explicitly claims to return an "approximation" of the header (which may or may not be a safe thing to use in different circumstances; but like, that in some sense isn't the API's fault?...).