If it was legitimate why would you need to stack the court. A legitimate court’s decisions would be accepted, not changed by adding judges.
And the reason why the idea to stack was defeated was because it was viewed as a run around judicial independence (e.g. making it less legitimate).
New Deal proponents didn’t like those rulings but there was no question that any of the judges hadn’t been properly appointed, nor was there a huge break with past precedent on the same issue.
We’re living in a time where a majority of the court are acting in blatantly politicized ways, interfering on behalf of their party to prevent fair elections and flipping alleged legal theories whenever necessary to produce their desired outcome.
That is incredibly risky for an institution which has considerable power on the grounds that it not play games like that.
The same can be said today. Supreme Court appointments have always been political. Hell being "Borked" got its name from political games during Reagan.
Just because you don't think the process was "proper" doesn't change the fact they were all confirmed by the senate, as per the "proper" process.
Funny how many people complain about the politics of Supreme Court nominees only when it works against their own political views.
If you want to do better, note that I was talking about the legitimacy of the court itself, which is critical to an institution with a great deal of inferred power which is not explicitly spelled out. The FDR-era court did not have a justice whose legitimacy was dubious. Similarly, you could look at the various New Deal rulings for examples which completely broke with long-standing precedent or relied on cherry-picked history if you want to argue that those rulings were not unusual.