I think your original understanding is correct - what we're calling "drivers" are run by the host OS and CPU to interact with a peripheral over PCI, USB, SPI, OWI, whatever. And "firmware" is some special software required by the device itself to run (could be some ARM or MIPS binary or whatever arch is used by CPU(s) on the device). I guess that some peripherals have firmware on non-volatile storage while others require it provided by the host when they're initialized.

So the "firmware" I required for my Thinkpad's wifi to work (and which was included in that Debian nonfree image) is this: https://wiki.debian.org/iwlwifi

update: actually on that Debian wiki there's a page called "Firmware" which explains things well https://wiki.debian.org/Firmware