Wittgenstein was not a constructivist, though. At least in his later years he explicitly argued against any -ism as a position and tried to avoid philosophical theses as a whole. (Whether he succeeded with this undogmatic approach is another question of course, but to say that he was a constructivist would commit him to a position that he did not advocate for, even if many of his remarks can certainly be read in such a way if they are read in isolation.)

I don't get the sense that Wittenstein was suspicious of infinity as an idea / concept in itself, only of its treatment in mathematical logic. Otherwise much of his writings on it could've been far more terse.