If it's true then it looks very suspicious that avg. latency decreases as the number of concurrent clients goes up. How do you measure avg. latency?

Let me illustrate what the avg. latency usually means.

Imagine that we have two concurrent users: Alice and Bob. During the testing lasting one second, Alice managed to send 3 requests per second, the first request took 0.5 seconds while the rest were executed in a quarter of a second each. In the same time, Bob did only 2 requests, each of them took 0.5 seconds.

So the average latency is the sum of execution time taken by each request divided by the number of all requests, in our case it's (0.5+0.25+0.25+0.5+0.5) / 5 = 0.4

Did you use the same definition?