Gnome is slow even on laptops, it was made for X and later ported to Wayland, it uses Clutter/Mutter, JavaScript, etc. So here yes definitively it's layers upon layers.
On the other hand, Posh is based on Wlroots and there is no layers. It's pure C code talking directly to EGL/DRM/KMS (And in the future Vulkan) stack.
Even more interesting, Wlroots allows the use of hardware layers and with that you directly write to the screen and compositing is done in hardware. You can't go more optimised than that.
With this kind of approach you can totally write a convergent, both phone & desktop, environment which is butter smooth on low spec mobile.
But of course then the whole question is which apps you run on that environment, the typical Electron app won't run smoothly (but it wouldn't either on a IPhone 1).
On the other hand, GTK 4 should be GPU accelerated and Qt too so those could run pretty smoothly
Phosh uses GTK, right? That's a pretty thick layer right there. You used to be able to put pixels on screen with a handful of ASM instructions https://www.cprogramming.com/tutorial/tut8.html
There are two parts : 'Phoc' and 'Posh'. Phoc is the Wayland compositor (Wlroots & C), Posh is the launcher UI (GTK).
Phosh is just "an app" (with special privileges to talk to Phoc). All it does is display app icons and windows thumbnails (which are not actual live windows but just screenshots of the apps (like on Android)) it does not composite windows, it does not stand between the apps and Phoc.
This is unlike Clutter & JavaScript which do stand between the apps and the Wayland compositor (Mutter) on GNOME.
So my point is that apps on the Librem talk directly to Phoc (not Posh) which is a super minimal layer and can draw pixels on screen with almost no overhead (and potentially none at all when they'll support hardware layers)
(Note : this is the Librem architecture as far as I understood, maybe I missed something)
It would not run at all. No mobile OS has a swap partition as far as I know, and it would not make sense to add one due to performance and power constraints.
The perceived slowness of the PinePhone is due to layers upon layers of software used to draw simple 2D stuff to the screen. I think a really simple bare metal GUI would be better than running a full blown linux desktop environment like GNOME. Even if you "skin" and strip it down for mobile, it is still laggy as hell.
Linux on a smartphone => yes
Linux desktop environment on a smartphone (even with Wayland support) => no
I am less interested in the "daily driver" part of the PinePhone and more interested in the power consumption and responsiveness of the device itself. Stuff like this is exciting (extending the standby time of a PinePhone to 100 hours): https://github.com/crust-firmware/crust