Semantics. An assembler parses a human-readable representation and emits an executable one. That's a "compiler" in most contexts, and certainly there have been many things called "compilers" that translate simpler things than general purpose assembly code.

Obviously the distinction being made is that "assemblers", even in the days of 8 bit CPUs (and earlier!) have always been able to translate at or near the speed of the I/O required to read the data. So there's an intuition that says they're "instantly fast" and thus deserve their own name.