Not necessarily — Congress has vast powers under the Fourteenth Amendment. See the just-published "Abolish the Senate" by lawyer and well-regarded liberal commentator Thomas Geohegan [0].
[0] https://thebaffler.com/salvos/abolish-the-senate-geoghegan
[1] https://newrepublic.com/article/158992/biden-trump-supreme-c...
The response to the slippery slope is not to keep on sliding.
Article I section 2 is the relevant part of the Constitution; it makes absolutely no statements about "The states are free to choose their way of allocating their House representatives so long as they are directly elected" and indeed that is absolutely not true; in fact the restriction to single member districts is a federal law passed by Congress, 2 USC 1 § 2c [1], after gerrymandering was producing multimember districts that were not state-wide in order to disenfrancise minorities.