Noise is anything that is still present when you take away the desired input signal, excluding deliberate time-based effects like reverbs and delays.

Frequency components that are not in the original signal, but are only present when the signal is there are usually distortion.

(Why "usually": radio-frequency interference is noise. In audio, you may not notice RF when there is no signal. But when signal is present, it may mix with RF in a nonlinear way, causing additional distortion products attributable to the RF. I.e. the RF noise is "descends" into the signal's frequency range via intermodulation.)

Noise would be thermal noise generated in circuit components, or mains hum, anything that is there when you aren't playing any notes. I wasn't spraking exclusively about guitar amps, all this applies to analog electronics in general.