Unless of course "VWO" (you should provide a link or descriptive text when mentioning a TLA we might not recognise, BTW) is somehow sensitive to what version other people use on their sites so you have to test against all versions.
You are right that we don't have to upgrade to every version but as you say performance improvement is sometimes very significant so we feel compelled to upgrade.
For most of your run-of-the-mill sites with a little sprinkling of jquery, to animate a few things and do some basic ajax, there isn't much point in upgrading. The performance gains will probably be negligible.
I've found when upgrading jQuery problems rarely occur, and with automated js testing + "vocal users" any bugs are known quickly.
...and of course your coding your site so if js fails it degrades gracefully into a usable site.
It's not a massive issue, I think.
Every time jQuery changes, it means extensive cross-browser testing for us agan.