The reverse situation would be the case (cable supports 3A, but not 6A). For the range of voltages supported by USB-PD, the cable only cares about amperage. Copper wire doesn’t care much if it’s 5V, 50V, or 250V, it’ll carry it the same. However, if the size of the wire isn’t correct for the amperage, it’ll over heat and potentially catch fire.

It’s the end device that cares about voltage and amperage both as the voltage steppers inside have to be ready to handle providing the right voltage to the chips and the amperage has to be right to handle the load.