There are many sets of non-transitive dice where the average is not the same though (I should have used one of those examples). Here's one:

   A: 4, 4, 4, 4, 0, 0 (avg: 8/3)
   B: 3, 3, 3, 3, 3, 3 (avg: 9/3)
   C: 6, 6, 2, 2, 2, 2 (avg: 10/3)
   D: 5, 5, 5, 1, 1, 1 (avg: 9/3)

There's no reason that the average values need to be the same to have non-transitive dice. To modify the original three to have the same winning properties, but radically different averages, just imagine doing this:

   Die A has sides 2, 2, 4, 4, 99, 99.
   Die B has sides 1, 1, 6, 6, 8, 8.
   Die C has sides 3, 3, 5, 5, 7, 7.

The expected value of an individual die roll doesn't play into it at all.

It doesn't play into this game because of the rounding. Again, you seem to be attempting to disagree with me, by echoing the exact same things I just said.

That's because the second person that disagrees with the same thing is expected to be more right on average.

In normal dice play expected value doesn't come into play. But if we were to do a long term sum of results (like amalcon proposed) then it expected value would come into play, and would determine the winner.

This illustrates the difference between premature rounding (normal dice play) and non premature rounding (long term summing of results).

But a long term sum of results is a completely different game. It doesn't matter what's on the face of the dice at all; all that matters is the average value per roll. It's not dice anymore! And you've defined the winning condition completely differently!

I also don't understand how normal dice play counts as "premature rounding". It's just how playing dice works -- you compare the numbers versus each other.

"But a long term sum of results is a completely different game."

That's exactly the point! A naive intuition about dice is that they have a single probabilistic long-term score. E.g. 6-sided die is 3.5. So if one dice beats another, it should be transitive (the intuition goes). It's the summation (implied by averaging over time) that leads to the naive intuition.

Maybe this description of the two games might help clarify the link.

Game 1: Player 1 chooses a dice, then player 2 chooses a dice. They both roll numbers, say a and b. Then player 2 gives player 1 (a - b) dollars.

Game 2: Player 1 chooses a dice, then player 2 chooses a dice. They both roll numbers, say a and b. Then player 2 gives player 1 (a - b > 0 ? 1 : -1) dollars.

You can kind of squint and see that game 2 is the same as game 1 just with (a - b) "rounded" to either 1 or -1.

You seem to be using "rounding" to mean "comparing the exact, unmodified results of individual rolls", which is probably the source of the confusion because that doesn't really correspond to any common definition of the word "rounding".

One way to interpret OP's 'rounding' concept is that in a round of a dice game, the winner's score is rounded up to 1, and the loser's score rounded down to 0. If, on the other hand, each turn each player got a fractional score between 0 and 1 (without 'rounding'), then that would represent a 'truer' evaluation of the dice.

The best way I can think of to allocate a fractional score to each die player is to sum all the dice rolled, and give each player their roll divided by the sum. So if players rolled 2 and 5, one player would get 2/7 and the other 5/7 - instead of rounding the scores to 0 and 1.

Note that to calculate the EV for a die under this scheme, you have to calculate it versus a particular opposing die, so it would produce a different EV for the transitive A/B/C dice in different combinations.

Perhaps that's what the OP was trying to suggest?

More or less this, except with a simpler way of allocating fractional scores: simply divide the value rolled on that die by the largest such value on any of the dice in the set (or, equivalently, round to either 0 or largest-such-value and skip the division step).