The "horror" is less about going to the chain restaurant, and more the derision you'd face from your peers at suggesting it. The logic goes "If you like shitty chain food, it's because you don't have good points of reference to compare against, because you haven't experienced a wide variety of cuisine, because you're poor".