I don't get the sense that Wittenstein was suspicious of infinity as an idea / concept in itself, only of its treatment in mathematical logic. Otherwise much of his writings on it could've been far more terse.