News for hackers. Stuff that matters.

A comment on URL Parsing in WebKit. Parent comment

Re:URL Parsing in WebKit by greglindahl on Friday December 02, 2016 @06:13PM (#13093123)
Google around for a "public suffix list" library for your favorite language, it will aid you in guessing that 'example.com' could be a valid FQDN. As a crawler guy, I see common suffix list features in the kinds of url libraries that I use or write.
[ Parent | on HN ]
Re:URL Parsing in WebKit by tokenizerrr on Friday December 02, 2016 @06:14PM (#13093134)
Don't forget to update your library whenever a new gTLD comes out
[ Parent | on HN ]
Re:URL Parsing in WebKit by greglindahl on Friday December 02, 2016 @07:46PM (#13093548)
It appears that most libraries download the list from Mozilla, but yes, you do need to ensure that it's downloaded frequently.
[ Parent | on HN ]