It's worse, Raspbian is compiled for armv6 to also support the BCM2835 (Raspberry Pi 1). Here's a performance comparison between a modern 64-bit armv8 Linux on a Le Potato SBC versus armv6 Raspbian on the Raspberry Pi 3:

https://libre.computer/2017/12/03/performance-and-power-cons...