> For example, adding an additional candidate should not affect the average score of the rest of the candidates, hence there is no problem with "independence of irrelevant alternatives".

It is non-obvious to me that rational voters in such a situation should not adjust their scores for the initial candidates. In fact, it seems like you should adjust scores in the prototypical situation where you prefer candidate A to candidate B, and detest candidate C. With only A and B present, you give A a high score and B a low score. When C enters the picture, you change your score for B to a high one, so that you make it maximally difficult for C to win.