Can confirm, had this problem the other day. CI started failing and after a short investigation I figured out sshd was just dropping my connections. Installed fail2ban first, but then just used a firewall since all valid IPs are known.