You can't easily, you have to debug them. Debugging intermittent test failures is hard, and you have no guarantee you're actually improving much tangible things despite the time invested in them. It just sucks.