Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by notpushkin on Thursday October 24, 2024 @12:54AM
(#41932036)
Most HTTP libraries would happily extract the result for you. [citation needed]
[ Parent
| on HN ]
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by throwaway2037 on Thursday October 24, 2024 @07:55AM
(#41934640)
Java class java.net.http.HttpClient
Python package requests
Whatever is the default these days in C#
Honestly, I have never used a modern HTTP client library that does not automatically decompress.
I guess libCurl might be a case where you need to add an option to force decompress.