Almost everything starts to make noises at higher currents (e.g. CPU voltage regulator). You have of course coils making noise, you have ceramic capacitors making noise to due piezoelectricity, even good old fashioned electrolyte caps can make noises and transistors buzz due to thermal expansion of the die under switching stress. Not in computers, but even batteries, cables and bus bars make noises under pulse current loads, as do film capacitors (used in high power snubbers).

In the case of capacitors it also goes the other way; tapping a capacitor induces voltage spikes which can disturb analogue signals.