Have you tried vanilla Emacs without any plugins (aka `emacs -Q`)? It's blazingly fast for me.
Doom emacs has a focus on speed that spacemacs just doesn't have. It's similar enough to spacemacs that it is easy to pick up if you got used to spacemacs at all. I'd recommend trying it out and see how it feels.
I'm sure at least one time spacemacs was partly to blame (aka ALL THE THINGS activated/installed) - but it's definitely a difference in "feel" where (Neo)Vim feels closer to vi, and emacs feel closer to vs code/Atom.