Technically the smallest addressable piece of memory is a word.

I don't think the term word has any consistent meaning. Certainly x86 doesn't use the term word to mean smallest addressable unit of memory. The x86 documentation defines a word as 16 bits, but x86 is byte addressable.

ARM is similar, ARM processors define a word as 32-bits, even on 64-bit ARM processors, but they are also byte addressable.

As best as I can tell, it seems like a word is whatever the size of the arithmetic or general purpose register is at the time that the processor was introduced, and even if later a new processor is introduced with larger registers, for backwards compatibility the size of a word remains the same.

Depends on your definition of addressable.

Lots of CISC architectures allow memory accesses in various units even if they call general-purpose-register-sized quantities "word".

Iirc the C standard specifies that all memory can be accessed via char*.

Every ISA I've ever used has used the term "word" to describe a 16- or 32-bit quantity, while having instructions to load and store individual bytes (8 bit quantities). I'm pretty sure you're straight up wrong here.

The difference between address A and address A+1 is one byte. By definition.

Some hardware may raise an exception if you attempt to retrieve a value at an address that is not a (greater than 1) multiple of a byte, but that has no bearing on the definition of a byte.

That's only true on a word-addressed machine; most CPUs are byte-addressed.

Which … if your heap always returns N bit aligned values, for some N … is there a name for that? The smallest heap addressable segment?