Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by mrtksn on Thursday October 24, 2024 @06:30AM
(#41934122)
What if they send a POST request?
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by meindnoch on Thursday October 24, 2024 @08:02AM
(#41934680)
You close the socket as soon as you see "POST" and there's no POST handler registered.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by mrtksn on Thursday October 24, 2024 @08:31AM
(#41934840)
what if you accept post for legit reasons?
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by meindnoch on Thursday October 24, 2024 @08:44AM
(#41934950)
Then you should use an HTTP library that lets you limit the uncompressed body size, and terminate the socket as soon as the limit is reached.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by quectophoton on Thursday October 24, 2024 @09:32AM
(#41935314)
This is a good moment to remind people to always check what are the size limits and timeouts of their servers (and any other relevant limits), no matter if they are libraries or applications. Never assume the defaults are good for you, because sometimes they aren't.
For example, as much as I'd like to praise Go's stdlib because of how far it can take you, the defaults for its HTTP server are "the URL+headers of a request can take as much as 1MB" and "no timeouts at all, wait forever if necessary". See the `Server` type and the `ListenAndServe` function (not method) from the `net/http` package for more details.
They're probably trying to be as conservative as possible, but 1MB is huge.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by chgs on Thursday October 24, 2024 @07:00AM
(#41934283)
Most servers will limit posts to a fairly low size by default.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by mrtksn on Thursday October 24, 2024 @07:05AM
(#41934317)
Yes but the idea behind zip bombs that they appear to be very small, when expanded it can be extremely large. Before attempting to decompress, the POST request may appear something like 20kb and end up being 20gb.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by Aeolun on Thursday October 24, 2024 @07:13AM
(#41934366)
I’d have to assume a decent implementation will eventually give up. It’s almost a given those will not be used by people crawling the internet for quick wins though.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by bell-cot on Thursday October 24, 2024 @10:35AM
(#41935913)
> I’d have to assume a decent implementation will ...
Just like all good c code always checks for overflows.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by neallindsay on Thursday October 24, 2024 @01:39PM
(#41937696)
GET requests usually don't have a body, but they can.
A GET request doesn't have a body. There's nothing to gzip.