But that doesn't matter. The reason we're careful about GMO technology is that there is a reasonable mechanism whereby harm may be the result.
We thus require careful testing and good quality science before allowing these technologies.
Unfortunately, politicians are often in the chain of "people granting permission", and many of them are stupid and ignorant and not interested in science.
You ask for a plausible mechanism for harm, but you also mention thalidomide. Thalidomide is a great example of unknown unknowns - we didn't know that chirality was important, and it was only after many birth defects that we found out.
So maybe there's something we don't know with GMOs, and we don't know that we don't know it so we can't reassure ourselves about it. We can do as much as we can to rule out those unknowns, and to reassure ourselves about the knowns.
I suppose I can understand Europe's arch-conservatism regarding food products after the thalidomide disaster, but I would certainly prefer to see actual evidence of harm from genetically-modified food, or even a plausible explanation of why a given GM food could cause injury and not just FUD FUD FUD.