s/mods/users? everyone can flag stories.

It's not flags (or not sufficient to remove the story):

https://news.ycombinator.com/item?id=39231055

Flags are sufficient. I just posted a comment on the comment you linked to: I have many times been the person who pressed "flag" on a story and then watched it immediately disappear.

I think there's some threshold of flags to upvotes and possibly some other metrics that determines whether a story vanishes, but flags can absolutely tip the scales.

I like Sam, but "$celebrity says $thing about $common-topic" is almost never a good basis for a frontpage thread on HN.

It's vital to HN that user flags and/or software like the flamewar detector clear most such submissions off the front page. They tend to attract a lot of upvotes because that's what sensational (and especially indignant-sensational) stories do. Without countervailing mechanisms, HN would be completely taken over by those stories.