I would imagine there are better solutions for the attachments now; in 2005 you couldn't really find that much to solve it as I remember.
I just mounted my ~/mail/attachments locally using SSHFS. Saving them with Mutt made them automatically "appear" on my machine. You could also install a few viewers on the server and then use X11 forwarding, which works fine if you have a decent connection.
I wonder when filesystems became good enough for maildir to work with your volume of emails.
'99-2001 I designed and deployed a webmail service where we handled ~2 million user accounts using Maildir's on a single SAN device formatted with ReiserFS (the fridge sized device in question was not much faster or higher capacity than my current laptop.... Actually the whole infrastructure was probably not much higher capacity than my laptop). Many of us working on it had individual folders with tens of thousands of messages.
"Pure" Maildir without the client doing additional indexing will quickly get very IO intensive, but storing hundreds of millions of small files in a single filesystem was a solved problem by then and with some simple indexing or caching its trivial to handle that volume.
We handled the indexing needed to reduce the IO load imposed by the clients by writing a custom POP server and delivery agent that maintained some more metadata - partially in the filename of the messages (read/unread status etc.) - saving us a stat() call per message on scanning the folders. Partially by maintaining cache files with even more details, saving us re-scanning the message headers (accessed by the web frontend via a couple of custom/non-standard POP commands - much easier than contemplating dealing with the horrors of IMAP; for actual non-web client access we put proxies in front of our "special" POP server).
I still have a soft spot for flat files when possible for that reason.
Also for our usage we saw a relational DB as a scalability hassle, as we needed to be prepared to scale this to many backends, and dealing with an RDBS when each set of objects was specific to a user was not very appealing. Today the challenge in that is much smaller due to faster hardware, and there are far more options in terms of database solutions.
But at the same time there's just not that much to gain.