As always it depends on what you want to measure. Comparing total checkout time including queuing though for 'speed (items/s)' is going to naturally favor the self-checkout because the choice between normal and self checkout is heavily weighted by the number of items a particular person has. Anecdotally I've never seen anyone go through self checkout with a full cart but see it all the time with the normal registers. Additionally there's rarely the same number of traditional registers actually staffed compared to the number of self checkouts.

I agree. I would say that a better measure than items/s would be customers/hr (closer to throughput rather than speed) averaged across an entire store with and without self-checkouts. As you said, the optimal setup will depend heavily on the mix of customer types ("express" vs. full cart) a particular store gets. There's probably a good ratio for most stores between the two and I think that's what they're all converging on now.

Agreed that space is a part of the comparison, but what I've seen is that although they put 3 checkouts in the y-axis space of a single staffed lane, they end up having to use up almost 3 times as much space in the x-axis to allow people to pass each other, which simply isn't necessary to allow in a staffed lane. So I don't believe the amount of space is that different.

No, there really is a huge difference. It's more like 1.5 times the y-axis space in your example, they could get 6 self service checkouts in the space of 2 staffed ones:

    >|1|   |2|   <    >|1|      |4|<
     | |   | |         |2|      |5|
     | |   | |         |3|      |6|

Or similar. What does slow things down is elderly or people apprehensive of technology, and tourists or others encountering them for the first time...

Space comes with a price too. If you ignore all costs, a traditional checkout that is read when you walk up is probably faster most of the time vs an available self-checkout.