Re:My url isn't your url
by kmike84 on Wednesday May 11, 2016 @10:13PM
(#11680897)
A good question; I'm not sure UTF-8 is always used because systems we tested were all using UTF-8 system encoding.
This question was not relevant for Scrapy because it makes sense to use UTF-8 as default regardless of what browsers do when page encoding is unknown; unlike encoding of URLs extracted from HTML this UTF-8 default doesn't lead to incompatible behavior.
This question was not relevant for Scrapy because it makes sense to use UTF-8 as default regardless of what browsers do when page encoding is unknown; unlike encoding of URLs extracted from HTML this UTF-8 default doesn't lead to incompatible behavior.