From their site: (http://www.on2.com/index.php?599)
And with no patent-pool royalty hassles, VP8 licensing terms are flexible and affordable to ensure maximum ROI potential.
So, it'd seem like just them.OTOH, I'm sure Google has thought about this problem and done some due diligence.
It's quite likely that any number of H.264 or other random patents apply just as well to VP8, especially given the USPTO's penchant for handing out patents that cover even the most basic of ideas. This is Apple's stated reason for refusing to implement Theora. In this vein, it should also be noted that there's no guarantee that licensing H.264 actually covers you for every patent.
I'd assume that if someone would be foolish enough to sue a h264 licensee, MPEG-LA (or it's members) would be quick to counter-sue them to oblivion.
There's a reason why patent owners just join the pool when they discover they're eligible for a nice easy income stream.
Equally, it's why productless patent trolls are so dangerous: since they don't make any products, such patent portfolios are useless as deterrence against them.