> Of the thousands of Pepes that I have unwillingly seen, none were neo-nazi except those in articles accusing Pepe of being a neo-nazi symbol.

YMMV. In the circles I browse, they are frequently used, but they were only used to troll. They are certainly used for more aggressive hate in other places, including on Twitter.