The states control both maps, the district map which determines the population eligible to elect the US Rep for a given district, and a separate district map (with more and smaller districts) that determines the population eligible to elect the State Rep for a given district.

Both are a problem. The latter just means that the State Congress can be artificially heavily tilted vs one party or the other.