Sure there are some anti-science people out there, but most people I've talked to are worried about pesticides.

GMO plants are designed to take much higher doses. That is a concern both environmentally and as a potential health issue.

One use if GMO is to engineer a plant, that is resistant to a specific herbicide, which is typically sold by the same company (see e.g. Monsanto and Glyphosate).

Another use is to engineer plants that produce insecticides themselves, which raises the concern if those insecticides affect other organisms as well.