First, that isnt a system of morality.

But moving past that I think you've missed some nuance on morals (or more generally social norms) - they help prevent the degenerate race to the bottom you're referencing. Its upsetting but pragmatic to nullify relative advantage by engaging in shitty behavior, but if you're frequently justifying behavior by citing hypothetical competitors then you're just shitting the place up for everyone by starting that race.