News for hackers. Stuff that matters.
Re:Mutt 1.7.0 released by wahern on Friday August 19, 2016 @10:46PM (#12324922)
Am I the only one who wishes Mutt loaded mbox files faster?

A couple of years ago I wrote an mbox parser that was several times faster than Mutt's. It really showed because at the time my server was running on a Soekris net6501, my mailbox was a few hundred megabytes, and my habit is to invoke and exit mutt repeatedly rather than using a dedicated terminal window. Since then I've been running my server on a Xeon E3, and it's less bothersome.

I investigated patching Mutt to use the parser, but it seemed a little too much trouble for the effort given how many one-liners have accreted in the existing implementation. Using an index would make things more responsive, too, but the faster parser could provide competitive responsiveness into the gigabyte territory while keeping things simple, in both code and on the filesystem.

Is this a common enough gripe that it might be accepted and appreciated? My parser used Ragel but I could implement it again in plain C so it didn't have a build-time dependency.

Re:Mutt 1.7.0 released by quesera on Saturday August 20, 2016 @02:38AM (#12325494)
I hate to make any assumptions about your environment, but surely you can abandon mbox for a modern storage format (e.g. Maildir)?

I can sympathize with the pain of a slow mbox parser, but even a magical parser can't resolve the fundamental flaws in the format.

Re:Mutt 1.7.0 released by _wmd on Saturday August 20, 2016 @05:53AM (#12325896)
Hey,

I'm not sure if it's upstreamed yet, but at least the Red Hat fork includes a patch to use unlocked libc IO which more than doubles the mbox parser performance: https://github.com/karelzak/mutt-kz/commit/0672404fc710b4af5...

Mutt's parser is woeful, it's a decade or so of hacks, and in some cases it can't even round-trip its own mboxes without corruption

Re:Mutt 1.7.0 released by notaplumber on Saturday August 20, 2016 @02:02AM (#12325425)
I'd wager any mbox optimization efforts are put on the back burner in lieu of modern usages, which seem to be in favor using it as a IMAP client only.
Re:Mutt 1.7.0 released by just_had_tea on Saturday August 20, 2016 @03:37AM (#12325613)
For years i used Mutt as an IMAP client, but that was honestly quite a nightmare. Even back in the day at uni with an über fast internet connection it was still just not responsive enough. The day i switched to using OfflineIMAP and a local Maildir it was as if the blinds had been drawn aside and sunshine streamed into my INBOX :).

Having said that, i use mu4e these days, because i've become an Emacs user. With a few tweaks it now scratches my itch very effectively (except of course the HTML-email problem as mentioned elsewhere -- this is especially troublesome at work where they want me to include an HTML-embedded-image-tabled monstrosity with about a page of legalese as a signature).

Re:Mutt 1.7.0 released by stevekemp on Saturday August 20, 2016 @02:30AM (#12325483)
I'd agree with that.

That said there is an obvious solution to slow MBOX folders, which wouldbe to switch to using Maildir-storage, then using the `header_cache` setting(s).

Re:Mutt 1.7.0 released by notaplumber on Saturday August 20, 2016 @02:48AM (#12325516)
I personally don't mind MBOX, it's a simple format and if you archive it regularly and don't let it grow exponentially, it can be nice to work with.
Re:Mutt 1.7.0 released by jabl on Saturday August 20, 2016 @07:27AM (#12326079)
mbox is fine for a read-only (or at least append-only) archive. It's when you start deleting messages, changing flags etc. in the middle that it becomes a pain.
Re:Mutt 1.7.0 released by dredmorbius on Sunday August 21, 2016 @05:18AM (#12330105)
Is there any particular reason you're using mbox rather than maildir (single-message-per-file)?

Mutt offers quite good header/index caching, and there are other tools for improving on that.

Re:Mutt 1.7.0 released by gkya on Saturday August 20, 2016 @05:19AM (#12325814)
The common solution is to split your mbox regularly. You can use mairix to search across many mboxes, it creates a new mbox with a certain name that contains the matching mail that you can open with mutt.
Re:Mutt 1.7.0 released by pilooch on Saturday August 20, 2016 @10:04AM (#12326643)
Weird, my mailboxes each contain 10K+ emails and load in a snap.