I think usenet can speak to the problem of trolls ruinging the community. Which is why web forms came to dominate, despite usenet's superiority in technical features and access (most people had email clients before web browsers).
Usenet was ruined by a small number of people who were trying to ruin others reputations by posting all sorts of insane diatribes to the group. Plus the spammers. Uggg... the spamemrs.
Your assumption is that it's impossible to differentiate category one from category two. I see no reason for that to be true.
Here's a question for you: If you moderate a list, do you let subscribers see what you've moderated and why? If it's for the benefit of the mailing list community, then why not?
If I did moderate a list (which I don't), and there was an ability for subscribers to see the raw feed, I'd allow that. I am curious if your approach will work. It's similar to how things operate here, but I wonder if it can work with no moderators. (HN does have editors who monitor flagged articles and comments.)
When you don't have to worry about if someone's being a troll or not real conversation happens.
This allows the creation of lists whose entire purpose is trolling each other (i.e. trolls trolling trolls trolling trolls...). Or in general if I think that the 'troll' flag is 'censorship,' then I have the option to disable people (on my list at least) from using it.
edit: fixed a typo
Generally there's three classes of trolls (this is just my opinion):
Griefers are what you describe. People who are more interested in gaming the implied social norms of the community in order to destroy it. These people suck, but totally blocking them eliminates the second class:
Dissenters are people who just disagree with you, and considering nearly every open source project is started by opinionated dissenters it's just bound to happen. I believe that without these people though your project dies because your ideas stagnate and you never really deal with actual criticism until it's too late. Also, they may just be the vocal part of your user base that wants something you don't.
Finally, moderators who use their moderation powers to "indirectly troll". By this I mean people who unilaterally deny people's communications because it would interfere with their hidden agendas. If you go look at nearly every failed community, at the heart is someone like this.
What I think, and what we'll find out, is that what really drives people away is not the presence of all of these, but the dominance of one of these types of "trolls". My idea is to just balance them out and give the community a real vote on what is what.