No, the ends are the device from which the backup is being taken, and the device to which the backup is restored: the start and the endpoints of the entire journey of the data.
When done properly, the places the data sits/transits in-between have no ability to read or decrypt the information; it's indistinguishable from random data to those intermediate storage/relay services or nodes, because they never have access to the keys to decrypt it.
When I'm backing up to an Apple server, I think of the two ends in the end-to-end encryption scheme as my computer and Apple's computer.