The vaccines themselves and vaccine scheduling change over time. How would we know if something harmful was introduced?

Unless of course as you state we are unwittingly participating in ad-hoc longitudinal studies.

There are any number of `modern` diseases with seemingly unknown causes. It's not a stretch to be skeptical of the safety of modern medicine in general.