The number of hypercalls isn't a good indicator of the attack surface area.
As examples:
There is a full x86 instruction decoder which is technically not part of the hypercall interface. x86 instruction encoding is surprisingly complicated. There was a vulnerability in that code: http://xenbits.xen.org/xsa/advisory-123.html
Handling x86 page tables and all of the various feature bits is also surprisingly complicated. This is part of the hypercall interface, but there was also a vulnerability in that code: http://xenbits.xen.org/xsa/advisory-148.html
The Xen attack surface is then reduced to about 20 hypercalls (even less for ARM). http://xenbits.xen.org/docs/unstable/hypercall/index.html