The VPx cross-license does not work that way.
You are granted patents to use VPx (you can not use these patents outside VPx scope) and you are granting patents necessary to use VPx[1]. If you have some other patents that are not necessary for the VPx, they are not subject of the cross-license.
[1] - "that must necessarily be infringed by the act of Exploiting Licensed Products in the Licensed Field of Use"
At the example of Pied Piper from SiliconValley: If they used VP.9 for video encoding before compressing and sending it, Google/hooli would now have the right to use their (potentially) patented algorithm for whatever they want, without ever getting sued.
Now that's great, eh?