I reckon filters were initially intended to remove specific frequencies/noise and the term grew from there. Which is how ffmpeg and other software ended up with video filters that don't actually remove anything.

Someone else mentioned lens filters as a possible source for the term but I'm pretty sure the signal processing term is at least partially responsible for the current usage.