Is there anything useful currently for anyone wanting to mess around with training an AI agent for a game they are developing?

I feel like in terms of game dev nothing supersedes state machines.

I know there's Gymnasium: https://gymnasium.farama.org/ and CleanRL and plan to mess with those but I havent seen much evidence they can be used to make a compelling adversary AI.