As netnichols stated above, "I had to wait about 15 minutes for 'the agent' to show up that would conduct the pat-down." I'm guessing that this is the reason.

I've successfully opted out once. In that instance, an agent performed a pat-down with no wait. I tried again more recently and, after fruitlessly waiting 5-10 minutes for the special pat-down agent to arrive, gave up and reluctantly went through the backscatter.