You are talking about data parallelism. Depending on the model tensor parallelism can still be very important for inference.