Which are presumably both useful to have within the same community. Maybe forum software should allow individual threads to be made flat or nested, at the discretion of the person creating the thread?

I could see that working, with the right community and moderators, especially if the choice were presented not in terms of simply flat-vs-nested, but "what is the goal of this discussion?" Those for informing and airing possibilities might have certain titling, features, and moderation (including nesting)... while those for hammering out some common experience/decision would have other titling, features, and moderation (including flatness).

(Remembering, as always, that it's not exclusively a goal for software design to solve, but also and probably primarily up to the participating people and their behavior... who can and do bend any offerings from the designers/coders to their own ends.)

How about:

Nesting: discussions converging towards separation, isolation, protectionism and flamewar ;)

Flat: discussions converging towards consent.

EDIT: Getting downvotes, but face the facts: HN, Reddit are pretty exceptional. Look at the legacy from the 90s and try to proof me wrong.

You can't just declare evidence against your claim "exceptional". Reddit has a large userbase, and includes numerous communities. Nested discussion seems to work pretty well for it.

Yes I can!

The point is this: reddit has a certain demographic that you don't find on most websites: mostly tech geeks that embrace modern web technology and a very high proportion of programmers. On the other hand look at slashdot, horrible discussions, just a random pile of opinions with many flame wars.

That said, discussions in News forums (news paper, news agencies etc) are often very stupid or unpolite. But I have seen that they are at least slightly more polite in flat forums. There is less repetition and if someone complains about unpoliteness, this is taken more seriously by the followups. Of course flat forums are way more work for the commentator because he needs to read all the posts that were before and make conclusions based on them. This can be next to impossible on an article's comment section with high load.

I think reddit's moderation system has more to do with the relative success of its discussions than its audience. Are you implying Slashdot didn't have a high proportion of programmers?

Yes, at least a lower proportion than Reddit. But maybe I'm wrong.