Also depends how many pages do you want to test with Selenium - if you want to test e.g 5 pages, then it'll be fast enough I guess.
There are a few tricks/configurations that make Selenium significantly faster - https://stackoverflow.com/a/57720610
A lot of is also on how do you write your "testing infrastructure" - maybe you could reuse engine/browser instance? execute tests parallelly?
There's also alternative to Selenium by Microsoft: https://github.com/microsoft/playwright
AFAIK it's way more reliable than Selenium (false errors)
__________
Overall you can still start application inside tests with different startup, without Selenium and send http requests, so you'll have end-to-end tests done fast.
The waste I am referring to is time it takes to get that feedback whilst the work is being actively developed.
A simple feature of a controller (to borrow someone else's example) is to change the response based on an attribute's configuration. It is wholly astonishing that one needs a _web-server_ to test that.
This way you can test those fancy things, but I agree it's kinda "hard" to get it working for the first time