In the Calico-without-ipip case, you're basically comparing host networking (let's assume with some iptables enabled) with host+iptables+bridge networking (yes, there may be more iptables rules involved). Let's assume the impact of iptables is the same between both, then it'd be interesting to measure the impact of the Linux bridge being used. As a colleague of mine mentioned, this may be barely noticeable on a 10Gb interface, but could be on faster networks. How about running these tests on a 20Gb or 40Gb network? These are quite common in datacenter networks, which is where you'll be using these CNIs (unlike GKE, EKS,... where you can integrate with the 'native' SDN).

Finally, I'd be interested to get some results when using the macvlan CNI plugin (though then one loses network policy support, sadly enough :( )