To add to that, email (and NNTP news) clients even from 20 or 30 years ago have other powerful features that web forums have yet to catch up on:
- kill files[1] (which you can use to filter out unwanted articles/mails based on content or metadata such as subject, user, etc)
- scoring
- user-configurable anti-spam filtering or other "intelligent" filtering (such as bayesian filtering not just for spam/ham, but for interesting/unintersting content)
- tagging not just on a site-wide level but at the client level so each user can tag messages/articles the way they make sense to them
- other advanced filtering and scripting based on any of the above
Web-based forums are just incredibly primitive compared to this many-decade-old technology.
Now, even though I and my email client still use threading, much of my incoming corpus does not. Instead, I see a mixture of users who break every thread (no In-Reply-To headers, ever), users who cannot comprehend subject metadata versus first line of message, and users who use the most recent few screens of activity as an address book, always replying to the end of a thread with a completely unrelated and new topic.
Or maybe give up and have some sort of manual threading overlay where I could rearrange and add my subject tags for messages..
It's a bit sad the code is messy and it's a tad slow compared to other options.
Notmuch for Emacs has so much potential to replace Gnus. The interface is incredibly fast and friendly (a classic tree view and a threaded view that IMHO surpasses Gmail). Plus super quick search.
I think it could become a killer application for Emacs (the other two now are org-mode and Magit IMHO).
However, there's no good tooling to keep Notmuch in two-way sync with maildirs right now. Everything works based on tags, which is incredibly flexible, but there should be some tooling to implement logic that allows moving mails around based on their tags, so that the remote IMAP store gets tidy.
People usually rely on scripts for simple use cases, which is fine albeit a bit hacky. Some DSL that resembles Sieve, but implemented in ELisp would be cool.
Notmuch doesn't have scoring, it doesn't have BBDB integration, it has no way to get arbitrary message headers from elisp, it only indexes a handful of headers.
I've heard it is possible to use notmuch as a backend for Gnus, to get the best of both worlds.
As for two-way sync, some people have come up ways of doing that, using something like Sieve.
For me two-way sync is not super critical, as I've come to view the webmail interface of my mail provider as just a backup to use in case my laptop dies or something. In all other cases, I'll just use notmuch, for which local tags are enough.
Scoring could be quite easy to implement with auto-tagging. Tags are a unifying concept in email. You can implement any custom user logic with tags. What notmuch is missing is a bit of tooling to do this. It'd be cool if this tooling had access to any arbitrary header, as you have pointed out.
I feel I don't need BBDB integration when I have autocompletion based on the email database. That said, there's bbdb-notmuch.el, and I assume other options as message-mode is really a Gnus thing.
I think basic two-way sync is almost always needed. Otherwise you end up with a huge inbox and even if you don't care at all about the remote IMAP store, mbsync calls will start slowing down.
If instead you search just your BBDB database, which only has the contacts you actually correspond with, you're going to get just the useful and relevant ones... and you'll always have the notmuch database to search on as a backup, in case the contact you want to to correspond with is not in BBDB yet.
Very sorry to hear that Gnus with a notmuch backend is slow, as that's what I was considering trying next.
I really like Notmuch's interface. I think limitations are perhaps not that hard to patch on Elisp unless you have a complicated setup.
I'm trying to implement a DSL to coordinate tag <-> maildir sync on top of a few macros.
Mu4e is fine, but I found the interface slightly slow and awkward compared to notmuch. Notmuch's interface is really really good and quick.
Mu4e doesn't ignore underlying folders, which is cool if you have a very simple workflow. It's also superior to notmuch in the sense tags are stored in message headers, so they can easily be sync'ed across clients.
If you buy Reddit gold which makes Reddit to highlight posts that are new since you last time viewed the discussion, then Reddit becomes almost as good as NNTP clients were 20 years ago.
[0]: https://eul.im
then i started actually handling the raw email messages. i got threads working pretty well but noticed frequent rendering issues and digging into the cause. my conclusion is, email threading is mostly a pipe dream; it can only be done properly if you have a view of the entire conversation (as is the case with centralized forums/message stores like HN).
the 'References:' header only represents the message ids of any particular branch up to the root. if you try turning a number of messages with the same root into proper threads, you quickly discover missing branches, partial branches/holes, missing leaf nodes, etc. If your knowledge of a thread starts with a message forwarded to you, no thread is possible (even though one clearly exists as indicated by the References chain in the header!). side-conversations that happen (selective replies, cc/bcc), additional fwd/reply chains are all invisible to you. sadly, an incomplete thread view is an extremely common thing (at least in my inbox/sent of ~15k messages).
the client can only thread the actual raw messages it has; those simply quoted in any replies or forwards (which established the Reference chains) don't count. the only commonality you have is knowing the root message id and the timestamp for ORDER BY. everything in between, you may or may not have to reconstruct a partial view of the thread...and each user will have a different view of that thread based on which parts they've been party to.
the threading that i did get working was nice, but not as useful as a bulletproof solution would have been. unfortunately, i don't believe that's possible with email.
i now understand why gmail does flat convos.
Them why has it been working so well for me since I started using email?
Maybe take a look at this for inspiration: https://fkref.com/?RbC_-NEE
the problem is with this step:
4. Prune empty containers.
you can prune/flatten these containers but then the deeper messages that you do have are not direct replies to their ancestors in the flattened/pruned version of the tree.
i didnt say that threading did not work. it does. but it is imperfect at best and can be misleading at worst.
in my case i was constructing an MPTT structure (Nested Set) for optimal thread fetching and simply leaving in the dummy message placeholders that i didnt have a copy of.
i plan to experiment further, but there arent really any earth-shaking revelations in that post. it's all fairly obvious.
Like so many other things, though, this only works if you enforce it. It's pretty horrifying what the GitHub generation tends to think passes for acceptable on bugtrackers, for example.
Or that earlier posts get lost because no one actually sees them. With threaded discussion, if one subthread has gone off-topic, you can always collapse or kill that subthread.
And your point doesn't always hold true. For example https://github.com/torvalds/linux/pull/17
This is a social problem. I deliberately added a disclaimer about the tendency for things to turn to crap if you tolerate it.
I don't know what this is supposed to mean:
> And your point doesn't always hold true. For example https://github.com/torvalds/linux/pull/17
> I don't know what this is supposed to mean
What I linked to is an example of a discussion that didn't stay focused or tight despite the fact that it's not threaded.
1. You shouldn't expect that it would "always hold true". I really don't know why you would, because I did not write that it would always hold true. In fact, what I did write is the opposite: a tacit acknowledgement that it's not a hard and fast guarantee. I mean, I specifically made comments like, "Like so many other things, this only works if you enforce it". It's right there. You can go back and read it. Thing is, I even edited my comment (before you read it and responded) from the original; I edited in the words "can help" so that it says "the unthreaded approach can help keep the discussion focused". You want to know something perverse? While doing this, I stopped and asked myself whether I was hedging too much—writing too defensively. Apparently the answer to that is a big nope; I was still not defensive enough.
2. Regardless of everything above, the page you linked to is supposed to prove what? It's a misdirected pull request, and the very first comment is a message to the requestor telling them so. I don't know what you expect past that; even after the message explaining that GitHub is not the correct place to make a pull request and that it will not be moving forward, you seem to be trying to score the discussion against a rubric of how effective the discussion was at staying focused. Do you not see what's wrong with that?
3. I made an a priori remark about how lousy the GitHub community is in my very first comment. I mean, my exact words were calling out what's found on GitHub as "pretty horrifying". If there's some way to be clearer regarding how I feel about GitHub, I don't know what it is. GitHub is terrible. And yet, despite this, despite everything above, your choice example is to link to a GitHub thread where this exact thing is on display, in an attempt to prove me wrong. Or something. I don't get it.
This'll be my last comment here. I think I'm gonna go away for a while.
Regarding the example I cited. it's an unthreaded discussion that has been going on for several years and, if you bother to read through it, it's a mixture of troll comments along with some informative comments/discussion. Had it been a properly threaded discussion, it would have been much easier to follow the informative thread branches and filter out the uninformative ones (where the classification of what falls under one category or the other is up to the person reading it).
But, because it's not threaded, you're effectively forced to wade through all the troll posts to find the informative comments (or miss the informative comments entirely if you're not willing to do that). The lack of threading did not change the "social aspect" of the discussion. It would have been the same either way given the topic, but one way of rendering it would have made it much easier to follow (and possibly spawned more informative discusson about the topic compared to what actually took place).
Getting back to your points:
> the very first comment is a message to the requestor telling them so. [...] after the message explaining that GitHub is not the correct place to make a pull request and that it will not be moving forward
If your assertion about unthreaded discussions was true, then it should have ended soon after that comment (maybe after a couple of comments from the person who submitted the pull request after Linus' comment). That obviously wasn't the case and nothing kept it in check.
> the unthreaded approach can help keep the discussion focused
Based on what I've read online over the years (both threaded and non-threaded discussions), it doesn't seem to be the case. The only way I think this could be settled would be to come up with a representative sample of discussions under both formats and how well they stay on the original topic compared to each other. It would be interesting to see the results of such an analysis.
GitHub and the like only work that well when there's maybe one or two things being discussed at once, because there's just a single linear stream of comments.