I'm curious what libraries you've come across from the game dev community that are useful. My usecase requires photo realism, consistent granular interaction, appropriate force response on contact with say a robotic arm. I've been checking out different tools and project Chrono seems to be the most accurate but also the most computationally expensive. I'm looking for something near real-time and haven't been able to find any - NVIDIA Isaac is meh and there's a new library called GranularGym that seems to be only written for a conference and never updated after that. Would love to get your thoughts/suggestions!

Have you looked at Gaea? There is a community version and it seems like it could be useful for exactly what you describe. https://quadspinner.com/