The problem with standard email storage methods is that they don't scale well. Mbox, the default in Thunderbird, creates large files, and if you backup changed files, eve… (read more)
The problem with standard email storage methods is that they don't scale well. Mbox, the default in Thunderbird, creates large files, and if you backup changed files, every backup requires recopying a multi-gigabyte file, because it has changed when even one message was added or deleted. Maildir, the alternative, puts each message in a separate file. But file systems have a lot of overhead and their own limits. When you have hundreds of thousands of messages, especially on a PC, that clutters the computer's file system, and between the file cluster size and directory overhead, it wastes a lot of space and slows down other operations. Two bad options. And let's not be too snarky about what that company from Redmond does in its awful mail system, with a giant proprietary mail database file that sometimes gets corrupted...
It is obviously a lot to ask to create a new mail file format that scales better. It would not be standard and could break some things anyway, though in the long run it might be a good idea. But I have an idea that might work for Thunderbird. As it stands now, it is possible to create archive files by month or year. Then new mail doesn't require those to be re-copied. WHAT IF a "virtual mailbox" could be created that, instead of consisting of one Mbox file or one maildir directory, consisted of a set of Mbox files treated by the UI as one? Quick Filter would filter them all as one, unlike with archives, and you could scroll through them all and sort them and see threads, but they'd be stored as monthly or annual (or whatever) files, the way archives are. Sending and receiving would be unchanged, using the current folders, but periodically the folder could have the oldest ones moved out and still made part of a virtual folder.
There are some modest virtual folder capabilities now, but I don't see how to do this. Does it seem like a possibility with what's there, or a modest enhancement, or is that the kind of thing that would be a huge effort?