Where have you seen this? Look at what vLLM or TensorRT will do on a 4090 at those batch sizes.