Re:Hidden latency in the Linux network stack
by creshal on Thursday April 07, 2016 @12:36PM
(#11448599)
> The enlarging of the hash table I think is less a fix for this problem (although it would halve the number of later connections being put in the bucket), and more just a good fix they happened to do at the same time.
Yes. It just reduces the risk that they run into this problem again with a different port constellation.
Re:Hidden latency in the Linux network stack
by stingraycharles on Thursday April 07, 2016 @02:33PM
(#11449466)
A bit pedantic, but it doesn't necessarily reduce the risk, but rather the impact: using 64 buckets would only have half as many connections go into a bad bucket. This, however, does not in any way decrease the chance of the problem occuring again.
Re:Hidden latency in the Linux network stack
by sinxoveretothex on Thursday April 07, 2016 @04:11PM
(#11450252)
Using twice as many buckets, there will be half as many destination ports in the same bucket (65535 / 32 ≈ 2048, 65535 / 64 ≈ 1024), but since the "bad" connections described in the blogpost all use the same destination port, it won't change anything wrt that.
It does, however, reduce the overall impact when all connections are considered.
Re:Hidden latency in the Linux network stack
by jcoffland on Thursday April 07, 2016 @04:17PM
(#11450304)
There is only a reduction in hash collisions if the destination ports are fairly evenly distributed. I just don't see how this helps at all in the case described.
Re:Hidden latency in the Linux network stack
by dhruvbird on Thursday April 07, 2016 @01:19PM
(#11448919)
@kbenson Your explanation makes a lot more sense. Increasing the hash table size probably didn't affect the perf. significantly, but binding to the ANY ip did.
Re:Hidden latency in the Linux network stack
by MBCook on Thursday April 07, 2016 @02:03PM
(#11449244)
So are things 'bound to star' checked somewhere else in the path? I don't understand why the hash table wouldn't just have entries for each of the receiving connections still.
Re:Hidden latency in the Linux network stack
by seabee on Thursday April 07, 2016 @02:08PM
(#11449286)
There is only one listener bound to star, instead of 16k listeners for every IP. Thus, the hash bucket mapping to port 53 has only one entry instead of 16k.
It works out the same for the application: 1 fd or 16k fds doesn't really matter if you're using epoll, and that single fd can accept connections to any of those 16k IP addresses.
I suspect that's the real fix. Now all those (16k) bound addresses aren't creating hash table entries, so other connections that happen to use a port that hashes to 21 (or 53 after enlarging the table) aren't being shoved into a hash bucket that starts with 16k entries already in it.
The enlarging of the hash table I think is less a fix for this problem (although it would halve the number of later connections being put in the bucket), and more just a good fix they happened to do at the same time.