Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by mrtksn on Thursday October 24, 2024 @05:30AM
(#41933776)
The gzip idea is giving me goosebumps however this must be a solved problem, right? I mean, the client device can also send zip bombs so it sounds like it should be DDOS 101?
[ Parent
| on HN ]
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by vasco on Thursday October 24, 2024 @07:39AM
(#41934538)
At least on the codebases I've worked on, having limits on time and size of any decompression that you do is something that quickly ends up in some internal utility library and nobody would dare directly uncompress anything. Way before you get zip bombs you usually get curious engineers noticing someone uploaded something a bit larger and that increased some average job time by a lot - which then gets fixed before you get big enough to attract zip bombs.
So a zip bomb would just decompress up to whatever internal limit and be discarded.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by meindnoch on Thursday October 24, 2024 @06:21AM
(#41934075)
>I mean, the client device can also send zip bombs
A GET request doesn't have a body. There's nothing to gzip.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by mrtksn on Thursday October 24, 2024 @06:30AM
(#41934122)
What if they send a POST request?
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by meindnoch on Thursday October 24, 2024 @08:02AM
(#41934680)
You close the socket as soon as you see "POST" and there's no POST handler registered.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by mrtksn on Thursday October 24, 2024 @08:31AM
(#41934840)
what if you accept post for legit reasons?
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by meindnoch on Thursday October 24, 2024 @08:44AM
(#41934950)
Then you should use an HTTP library that lets you limit the uncompressed body size, and terminate the socket as soon as the limit is reached.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by quectophoton on Thursday October 24, 2024 @09:32AM
(#41935314)
This is a good moment to remind people to always check what are the size limits and timeouts of their servers (and any other relevant limits), no matter if they are libraries or applications. Never assume the defaults are good for you, because sometimes they aren't.
For example, as much as I'd like to praise Go's stdlib because of how far it can take you, the defaults for its HTTP server are "the URL+headers of a request can take as much as 1MB" and "no timeouts at all, wait forever if necessary". See the `Server` type and the `ListenAndServe` function (not method) from the `net/http` package for more details.
They're probably trying to be as conservative as possible, but 1MB is huge.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by chgs on Thursday October 24, 2024 @07:00AM
(#41934283)
Most servers will limit posts to a fairly low size by default.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by mrtksn on Thursday October 24, 2024 @07:05AM
(#41934317)
Yes but the idea behind zip bombs that they appear to be very small, when expanded it can be extremely large. Before attempting to decompress, the POST request may appear something like 20kb and end up being 20gb.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by Aeolun on Thursday October 24, 2024 @07:13AM
(#41934366)
I’d have to assume a decent implementation will eventually give up. It’s almost a given those will not be used by people crawling the internet for quick wins though.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by bell-cot on Thursday October 24, 2024 @10:35AM
(#41935913)
> I’d have to assume a decent implementation will ...
Just like all good c code always checks for overflows.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots
by neallindsay on Thursday October 24, 2024 @01:39PM
(#41937696)
GET requests usually don't have a body, but they can.