That only makes sense if we label foods with every step of their production and the labels would become large documents. There is nothing special about GM which warrants a special label.

GM is probably much safer than mutation via massive radiation - but nobody is asking to label that.

Why is "whether its use is safe or dangerous" not a science question?

I think GP's view is this: science is about expanding the space of possible choices. Business is about making the choices. "Safe" and "dangerous" are adjectives referring to concrete choices in context of their application, therefore a business question.

I believe this view is correct, and I share it, but with a caveat that labels are flexible and context-dependent. People believing GM is itself dangerous and bad are just victims of ideology. But other people saying GM is dangerous actually mean concrete instances of applying GM methods - i.e. they're really criticizing a bunch of business decisions, but doing it in an ambiguous way. Therefore, for the cases where you actually care about other person's opinions, it's worth to dig deeper into what they actually mean and why.