Raspberry Pi 3 was a Cortex-A53 back ported to 40nm outright.

Raspberry Pi 4 is on 28nm. (for comparison, Apple A8 and Snapdragon 810 were on 20nm already, in 2014-15)

It runs with a Cortex-A72 at a low clock (1.5GHz, phones even are at twice that clock nowadays) and quite high power consumption because of the process node.

The memory interface is narrow (32-bit bus, phones ship with a 64-bit bus and laptops/desktops with a 128b one) at a low data rate (LPDDR4-3200).

In addition to that, the CPU can only use 5GB/sec of it, with the remainder being reserved for the GPU only.

Those were some of the sacrifices needed to reach this price point. (and why it isn't representative at all of the performance of higher-end ARMs)

At the same time Intel was heavily subsidizing their x86 mobile SoCs to tablet manufacturers and for a while it was actually quite competitive price wise but it ultimately went bothered.