You can play games you bought in the Epic store, and are usually launched with the Epic launcher, without having the Epic launcher installed??

It depends on the game, but if they require the Epic launcher it will automatically open the Epic launcher and run the game for you. No need to keep it always running or search multiple store launchers to find which store you purchased X game with. They're all in one place.

https://heroicgameslauncher.com/ seems to work well on my steam deck.

I'd also recommend https://playnite.link/ which is an open source launcher that does the same stuff.

That's really fantastic. Better integration than the GOG client, and it isn't always trying to sell me something!