Yea, I agree with your interpretation on the root cause of the disagreement. I don't mean to weigh in one way or the other, but I would like to point out that people's mathematical intuition when it comes to infinity or obscenely large numbers can be arbitrarily wrong. It takes a lot of training to even recognize the mistakes you're making, let alone correct for them.
The most common example I see is the dust-specks vs torture thing[1], where Eliezer thinks increasing dust specks eventually outweighs the torture and various other people think torture outweighs any number of dust specks. Eliezer is basing his position on taking a few decision theory axioms as obviously true, and others are basing their position on taking their moral intuitions as obviously true (i.e. a "One's modus ponens is another's modus tollens" situation).
1: http://lesswrong.com/lw/kn/torture_vs_dust_specks/