We do indeed use Unity! Somehow I was aware of ml-agents, but didn't realize it was a "gymification" library. I thought it was a framework for running stuff within the Unity editor, but seeing that it allows things to be run from a normal python environment, that's awesome. Thanks for making me look at the README and documentation closer.
And here I was thinking I would need to write a native dll wrapper or wait till my next game project to include necessary bits from the outside.
https://github.com/Unity-Technologies/ml-agents
It makes it really easy to make games for reinforcement learning. I worked on it a bit over summer, so if you have any questions, feel free to reach out to me.