I would start with David Silvers (DeepMind) youtube series to get an idea of what's possible or not.

Running an already trained reinforcement learning agent is relatively cheap (unless your model is massive).

I suspect the reason people aren't using it yet is because it's a) really difficult to get right in training, even basic convergence is not guaranteed without careful tuning b) really difficult to guarantee reasonable behavior outside of the scenarios you're able to reach in QA.

edit: Link to lecture series https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLqYmG7hTra...

There‘s a really old Game AI book by O‘Reilly with a lot of clever simple solutions. You dont need the most complex, innovative and state of the art algorithms for games, most games have a really simple ai concept.

Game AI has very little, if anything at all other than name, in common with Machine Learning AI.

The answer is dont touch ML, start with FSM.

Start by making agents by using heurists and not AI. Once you have these in your game at least you can release it. If you are not happy with that performance then you can start to look at RL methods.