It's absolutely noticeable. The question is whether it's tolerable, and people do seem to tolerate increasing latency both from networks and their TV.

But, personally, I find the experience much less enjoyable as the latency goes up. It's not about being competitive either. (I don't play games online.) It just feels sticky and sluggish and I don't enjoy it. I miss the crisp responsiveness of older consoles. :(