I think mostly what's behind anti-GMO attitudes is the fear of unknown and unintended consequences. We've seen a lot of those ever since technology has been invented and they do seem to have proliferated in the last couple of centuries. So, it's not unjustified, especially when messing with extremely complex and barely understood systems. It's the same for weather engineering, for example. Parable of the sorcerer's apprentice and all that...
There are strong arguments against, not necessarily GMO as a technology, but practically all existing commercial uses of it. Furthermore, each use of it is potentially an independent problem with its own dangers. Thus far the only beneficial use of it I have heard of is "yellow rice", which is not actually available, for purely capitalistic reasons.
Also there's a lot more to GMO foods than yellow rice, so it seems you're proving my point quite nicely.
Maybe you don't think the benefits outweigh the risks, and that's a valid question.
However, the concept obviously has tremendous potential. All the variety of life is after all genetics. Improvements in nitrogen uptake could save tremendous amounts of fertilizer and help the environment, for example.