All else equal, twice the voltage yields twice the current (and 4x the power).
So while yes, it's the current that kills you, in the majority of circumstances high voltage = high current. Outside of explicitly current-limited setups (e.g. a Van de Graaff generator or a desktop ionizer), most higher voltage devices will kill you more easily.
And so the "voltage doesn't kill" meme more often than not (IMO) gives laymen an undue sense of safety when working with higher voltages, at the benefit of appealing to the more electrically-inclined crowds sense of "being precisely correct".
Just look at the dozens deaths a year from people messing with microwave transformers, trying to make some pretty wood lightning art, for an example of this.
I = V/R
It's as simple as that. A high enough voltage will find a current path ionizing the air or whatever is in it's way. A high current doesn't doesn't do that by itself.
That's why you see warning signs for "HIGH VOLTAGE" but not "HIGH CURRENT".
What does the capacitor supply, that the PSU can't? Could it be discharge current?
Before you accuse me of repeating dangerous tropes, please consider making sure to discuss it with the folks at VDE, because this is what it says in their (and the equivalent european) product norms and norms for electrical installations.
When touching the positive pole of a 1V 1000A PSU (I assume DC) you won't have 1000A flowing through your body. It is not the rating current of the PSU that kills you, but the actual current going through your body. And how well that current will flow through your body depends on many things, including voltage, frequency (if we are talking AC), contact resistances, skin resistance, the internal resistance of your body, how big you are, your ground connection resistance, the duration of the current flow, etc.
There is a reason we use high AC voltage for transportation of currents for long distances.