Four words: "You shouldn't have to".
Unfortunately, due this responsibility being approximately 5% of my time, I don't have time to invest in to-and-fro with Atlassian. I expect something to work (ironically like Trac did - its predecessor).
On the positive side, we're upgrading to latest versions soon.
Get onto Atlassian and get them to help you (at least with your slowness and crash issues) - their support is good once you get them engaged.
I've had all the problems you describe (large instance - 100k issues, internet facing) - sometimes it's as simple as a resetting a bad VTL template. They will get to the bottom of it.