And by and large it's not like centre- or even left-wing parties are in favour of "all refugees welcome" or "[all] immigration is good". Actually Netherlands has had a broadly right-wing government for about 20 years now, which has generally been against immigration.
And if we look at France, Germany, UK, then the situation is roughly similar (with differences in details and nuance, of course).
If I take the example of the UK: arguably right-wing government for 14 years with grass root Conservative backers against immigration. Well, last year the UK had the highest legal immigration on record.
Considering the popularity of Wilders in the Netherlands perhaps the government did not actually do anything to limit non-EU immigration, either...
If the people feel taken for a ride this cannot end well.
Somehow the mainstream parties and especially the left have shifted from anti-racism to a blanket and absolute "immigration is good" and "all refugees welcome" consensus (and not only that but in some cases a stance that more diveristy should actually be encouraged) and anyone raising concerns is indeed labelled as "right-wing" or "far-right", which is not helpful at all.