If you want to change your diet for environmental reasons, I would recommend going for net positive rather than vegetarianism. It cost a bit extra, but products like honey (bee farming is generally seen as a net positive on the environment), fishing from overpopulated lakes, and meat/plants from invasive species are examples where additional commercial pressure would do wonders.

Invasive species in general often carries the problem that hunting them cost more than the population is willing to pay, so they are "allowed" to spread and cause havoc with the ecology. Overpopulated lakes generally stay such because the fish is seen as undesirable, often based purely on local cultural views. Vegetarianism do not address those issue and generally the veggies being grow are those that is already being grown beyond what would be healthy from a bio diversity perspective.

While the effect of livestock is major, my concerns were eating plastic fibres in fish (though the negative impacts of aquaculture are many) and suppporting cartels by eating avocados.