Has Thunderbird destroyed all my email?
My email (IMAP) server has just moved and I went into Server Settings to set up the new server info. As soon as I did, Thunderbird appears to have deleted not only all local copies of my IMAP mail, but also ALL of my Local Folders.
Looking in my (Win10) Local and Roaming profile directories, I can see lots of *.msf index files, but all data files, except my INBOX, appear to be gone.
Looking at Activity Manager (see attached) it looks like that's exactly what TBird did. Delete everything.
Any suggestions as to how to recover from this, or am I just hosed?
(yes, yes, restore from backup...any OTHER suggestions?)
Also, WHY would TBird ever do this? Surely this (delete all mail without warning or notification) is not intended behavior?
All Replies (6)
When you use IMAP the messages are still on the server of
your mail provider.
IMAP delivers a synchronisation between your provider and
your computer.
The synch process is stopped now, so your do not see the messages on
your computer any more.
To put a finer point on what Johann wrote, what Thunderbird keeps locally for the imap account is precisely what is on the server. If the server changes, or your account on the server changes, then so does the local copy in Thunderbird for that account. That often means deleting messages to match what is on the server.
However, email in "Local Folders" should be unaffected.
Johann and Wayne,
Thanks for responding, but I feel like you didn't read my post very carefully.
When I changed the settings for my IMAP account, all local copies of my IMAP mail were deleted. I did not expect this, but it's not that big a deal, since, as Johann pointed out, my mail should still be on the server (it actually isn't but that's a whole other story, and it's in an archive on the server...it just isn't online any more.). Note that it isn't just that IMAP isn't syncing with the server any more, the local mbox files are gone. I should note that the account was set to never delete messages when messages were deleted on the server, which I expected to leave the local copies alone, but it's all on the server so whatever.
So, odd behavior, IMO, but fine.
More concerning, is that, as I said, Thunderbird also deleted all of my local files. All of them. The only two mbox files that are left in the Local Folders directory of my profile are "Trash" and "Unsent Messages", and they are both empty (0 KB...image attached). This directory used to have dozens of folders and GB of mail.
At first I thought maybe I was looking in the wrong place, or TBird had started storing it's mail somewhere else and moved the files. Both of these have happened to me in the past, and while it's pretty scary when your mail disappears, I've been through that and expected to just have to rebuild the indexes or something and everything would be fine. But the files were GONE.
I can't think of any reason TBird would do this on purpose, so I assumed I must be missing something, hence my first post. But after extensive searches of my local drive (hours long...I looked *everywhere*) I found nothing. I even did full text searches for things I knew were in the mail and unlikely to be anywhere else. Finally I started doing data recovery type work, and managed to recover some of the files using TestDisk (which is a forensic data recovery tool).
Thunderbird really did delete all my Local Folders. I have no idea why. I wasn't able to recover everything, so I'll end up digging out old backups. Fortunately (?) while my backups are really old, this is all really old mail, and all the valuable stuff should be in the backups.
I guess my problem is "solved" for the moment, since no longer sounds like I'm missing something. But I'm very concerned that I've hit some kind of bug or strange edge case that needs further investigation.
I don't have time right now (thanks to my long post here...sorry :) ) but I think I'll try to work up a bug report in the near future and post it to Bugzilla. (assuming that's still the right place)
I think your expectations and reality differ. So you ask folk to read you original post, instead of getting the facts.
IMAP is synchronized to the server. You changed the server settings to new details, so Thunderbird synchronized your mail, just as you specified, except the specification was to an empty serer. Now you have an empty mail account.
There is no real "local copies" in IMAP. You can export mail from an IMAP account either to a local folder within Thunderbird, or using an add-on such as the import export tools https://addons.thunderbird.net/en-US/thunderbird/addon/importexporttools-ng/?src=ss But unless you take some action you do not have a local copy. What you have is a synced copy that has about the same value as the contents of your Firefox cache. Exactly nothing unless you count the time savings having a local cache provides.
So what we are dealing with here is an issue that is almost certainly of expectations V's the realities of the IMAP protocol.
IMAP has the ability to store mail for offline use. This does generate a full local cache, but it also has the option to not have a local copy, and based on your posts here and past recovery attempts. Unless something fundamental has changed, you had the no local offline cache selected, in which case you would have basically an MSF file for each folder which it appears you have located.
Normally there are also mbox file (they have no extension BTW and live in the same folder as the MSF) that contains all of the local mail copies, but with their status set to deleted. This forum is full of advise to people on how to recover those deleted mails from the mbox files.
Not wishing to set up a scenario just as yours to determine if there has been a fundamental change to Thunderbird, I really can not be certain. But I would have though such a change would have bubbled to the top on at least one of the Thunderbird communication forums I monitor.
Matt,
Again, thanks for replying, but you still don't seem to be hearing me. Maybe I'm not saying it right. I understand how IMAP works. I understand that the local copies of mails in IMAP accounts are synced with the server. I have IMPLEMENTED IMAP servers.
What I'm saying is that when I changed the IMAP settings, Thunderbird deleted all my local IMAP folders. I didn't really expect this, as I thought I had set things up for it not to do that, but I totally understand why that happened.
But Thunderbird ALSO deleted everything under "Local Folders". It doesn't seem to me that that should ever happen, and yet it did.
I have filed a bug report on Bugzilla here: https://bugzilla.mozilla.org/show_bug.cgi?id=1577678
And as Wayne said in your bug, the code is in no way linked. So the culprit for the mess is unlikely to be Thunderbird.
However what is the problem? You changed the server name. Was the mail already copied to the new IMAP server? My assumption has been no because if it was the mail would have synced from the new server already, so deletion of all folders in the IMAP account would be expected. I must discount this as normal synchronisation behaviour.
So we are left with the removal of mail from your local folders. This is not shown in the images you post, so what caused this. In the bug you assert that the change was accompanied by a request to compact. This I would expect as all those IMAP folders represent significant space savings for compaction to kick in.
Thunderbird maintains local folders, like POP mail accounts using different code to that used for IMAP accounts, hence the mail and IMAPMail folders in the profile, and the inability to change an IMAP account to POP and vise versa.
You mention in the bug that you were prompted to compact. That does put a different light on the subject. One that I find usually ends at the door of the installed anti virus product.
Compacting is a two stage process with essentially three steps.
- First a new file is written to hold all the mail, this has all of the "non deleted" mail in the existing folder written to it. 'using the MSF index as a primary source of the current mail.
- The second step is the deletion of the exiting file
- Step three is to rename the newly written data file (nstmp) to the original file name.
The major issue with this process is that when you have other "things" in the background the copy and rename can be mangled. Some common causes are;
- Scanning by an anti virus program. Nothing like contention to mess up file writes in a product that assumes the operating system is actually in control. We do recommend creating an exception in the AV product to prevent on access scanning in the profile folder.
- Using cloud or streaming backup to "backup" files either to the cloud or local storage device. Files that are accessed often and are also large are prone to both Sync failures and interruptions that cause corruption in that type of scenario.
- Unreliable disk media. Because Thunderbird maintains such large files and is also regularly updating most of them (appending data usually) it is at the pointy end of any pending failure of the physical media. Especially when the compact starts allocation 4 or 5 Gb of space for a new inbox.
These issues are compounded when the mail files have been deleted. This does occur, usually again by anti virus products. (I am still waiting for someone to offer a reproducible action in Thunderbird to demonstrate an internal cause. My wait has been more than 5 years so far.) The anti virus detects "a threat" in the file and deletes it. Thunderbird does not really notice the file missing so the index continues to be displayed in the user interface, but when compact comes round, the missing file means that compacting appear to remove mail.
So I have nothing to offer as a cause, except to doubt that the cause is Thunderbird were your local folders are concerned.