It definitely needs to be thoroughly studied. But to my understanding, the past decades of study into it haven't revealed any harmful effects so far, at the energy levels that humans are generally exposed to. I do think it should continue to be studied in different ways, but given the current knowledge, I don't think people should feel very concerned about it.
To preface: I don't think there's any evidence at this time that standard levels of non-ionizing electromagnetic radiation are harmful to humans.
But it's not inconceivable that there could be some harm caused which we're not aware of yet and which is eventually discovered in the future. The body and brain are extremely complex and not fully understood, etc. Molecular biology is sensitive. It's not implausible that there could hypothetically be certain chemical reactions or cell processes which are subtly disrupted by certain frequency bands at certain amplitudes.
I personally don't think there will be such a finding, but people still need to keep an open mind, I think.
>We've reversed course on plenty of things that were once believed to be fine
That is my concern. Specifically that the "non-ionizing radiation" is a red-herring. Its possible that EMF exposure effects cells in another way that we've yet to realize. As a poor analogy: Arguing that being exposed to constant visible light on the basis that "Light is non-ionizing" misses the fact that constant visible light disrupts the circadian rhythm which has a host of other ill-effects.
To be clear, I'm not suggesting that we ban something on a fear, but given the enormous potential cost if we get it wrong, it should be thoroughly studied.