I had a very similar problem recently with docker + nginx. Best I could figure out was the randomness of the problem was being caused by keep alive connection limits. If the connection was opened as IPv4 it would work until it hit the keep alive limit but the new connection might run into the IPv4/IPv6 lookup problem and fail. Never really figured it out for sure. It's definitely thrown some cold water on my plans to go dual stack everywhere all the time. Not sure it's worth the risk of running into these stupid bugs.