News for hackers. Stuff that matters.
Re:Maintaining 65k open connections in a single Ruby process by WJW on Monday October 29, 2018 @06:35PM (#18332237)
Can you explain more how you get to 2^48? I thought the protocol specification only reserved two bytes for the port and so getting over 65536 connections would either require using more IP adresses or more ports. This seemed supported by answers like https://www.quora.com/What-is-the-maximum-number-of-concurre.... If you can enlighten me, I'd be much obliged.

You are definitely right that this is not a "full" solution for the c10k problem, but it is an interesting start IMO. Maybe for next month I'll implement basic epoll functionality to make an echo server or something :)

Re:Maintaining 65k open connections in a single Ruby process by toast0 on Monday October 29, 2018 @06:57PM (#18332408)
If I'm a server, and my ip is 192.0.2.1, and i listen on port 80, my nearby clients 192.0.2.2 and 192.0.2.3 can each establish just about 64k connections to me (port 0 doesn't generally work). I like to think of it as a 5-tuple

{Protocol, PeerA, PeerB, PortA, PortB} ex:

{tcp, 192.0.2.1, 192.0.2.2, 80, 12345} is distinct from {tcp, 192.0.2.1, 192.0.2.3, 80, 12345}

With roughly 2^32 ipv4 addresses and 2^16 client ports for each ip, you should be able to establish 2^48 connections to your server IP. Although, it usually requires a bit more than standard effort to actually use quite that many client ports per IP, by default you have to fight against high vs low ports, and anyway not a lot of OSes have a very sane allocation policy for ports.

Note also: if you're testing this on localhost, IANA has assigned you 2^24 ipv4 addresses for localhost, 127.0.0.0/8, so if i assume you're going to listen on all addresses, and connect to all addresses from all addresses, that gives you 2^24 x ( 2^24 x 2^16 ) = 2^64 connections. It might take you some time to iterate through an array that large :)

Re:Maintaining 65k open connections in a single Ruby process by masklinn on Monday October 29, 2018 @06:36PM (#18332239)
> Can you explain more how you get to 2^48? I thought the protocol specification only reserved two bytes for the port and so getting over 65536 connections would either require using more IP adresses or more ports.

32b IP * 16 bit port?

> You are definitely right that this is not a "full" solution for the c10k problem, but it is an interesting start IMO.

It's not a solution to the C10K at all, because the C10K problem is outdated and was trivialised a decade back. Whatsapp was doing 2 million connections on a single box (with resources to spare) back in 2012: https://blog.whatsapp.com/196/1-million-is-so-2011 and folks were working on C10M: http://highscalability.com/blog/2013/5/13/the-secret-to-10-m...

> Maybe for next month I'll implement basic epoll functionality to make an echo server or something :)

Epoll is bad, why would you want to do that?

Re:Maintaining 65k open connections in a single Ruby process by meguest on Monday October 29, 2018 @08:18PM (#18332914)
> epoll is bad

Source please! I'm genuine - what is bad about it?

Re:Maintaining 65k open connections in a single Ruby process by masklinn on Tuesday October 30, 2018 @05:04AM (#18334890)
Re:Maintaining 65k open connections in a single Ruby process by kjeetgill on Tuesday October 30, 2018 @01:24AM (#18334155)
It comes up all the time but I'm not qualified to speak to it off the top of my head.

Google comparisons with kqueue from BSD and iocp on Windows.