That's the thing though, I never hear that statement from conservatives or non-liberals - they never seem to utter it, even in a sarcastic or mocking tone.

I only ever hear it from what seem to be sincere liberals/progressives, and often in the context of dismissing claims of liberal bias. In fact, that seems to be exactly how it was used in this very comment chain.