As someone who's pretty idle, the worst (and only bad) part is everybody telling you how something must be wrong. Like people can get outright hostile and say pretty mean things. Everybody's completely fine with you not doing much, but if you actually say you're not doing anything instead of coming up with a random answer they go nuts!

Which I must admit I enjoy in a weird way.