I suspect your case is not that the packets are simply dropped but that the TCP connections are reset. Lookup your TCP stack statistics to verify. If that's the case try to find out if the resets are made by your side, the source or the intermediaries.