Yes - it seems amazing that 90+% of web forums are the old php/vBulletin type. My major gripe with these is that they do not have threaded comments. Not to mention a very old-skool UI/UX as you say.

Interesting what you say about db ownership. And I guess for established forums there would be migration considerations too (maybe this is why they haven't upgraded?)

Customization would be no problem. You could allow users to source their own css for example.

Some anecdotes from running multiple forum sites over the years:

Most forums end up using flat replies because normal people don't understand threaded replies. A friend at YouTube recently said that their research backed up that anecdotal assumption.

A couple of the major forum vendors tried offering hosted solutions and it wasn't sustainable as a business model for them; they both went back to simply licensing the forum software itself.

Forum communities are like any other social system: network effects and "where your friends are" are what keep you from leaving. Most "power user" features go unused by the majority. Stats like only 3% in one community use the friends list, only 10% in another use the profiles, etc. It's never about the feature set, it's all about the community.

Existing forum software is basically a black box. It's hard to integrate with and expand. It's hands-off for the admin until they realize they want a site that's more than just a forum.

It's interesting what you say about flat replies. Although a "technical" person, I can't believe that users are not wanting to reply to individual posts. Perhaps it's a UI issue?

There are certain social dynamics that have grown up around this variety of single-thread-per-post forum: discussion is channeled into essentially one discussion per post (you start a new post if you want to fork it); threads are "bumped" when someone replies and the expectation is that it's a reply to the main thread of conversation; the pagination encourages discussion to move on (in a 50-page thread, you don't normally belatedly reply to something on page 8, unless you have a good reason for it); etc. Threaded discussions are very different in style imo, and have less of a "conversation" feel.

I tend to prefer threaded overall, but there are some non-threaded forums I'm on where I don't think the community would work as well with threaded. For example, I don't think the SomethingAwful forums would work well at all as a Slashdot-style (K5-style, HN-style, reddit-style) threaded forum. Actually, I think some parts of reddit would work better with unthreaded; there's often a sort of urban-sprawl feel to the thread branching there.