Hi again

This problem is getting worse - my Inbox is now missing all emails between 10 July and 6 August. I appreciate the immediate response is likely to be "you must have accidentally moved them". I have tried searching across the whole of my Local Folder for these, to no apparent avail. A global search for a known email address runs for several hours without completing.

There is also an odd issue when I try to compact the folders - I get a message saying the "Temp" folder cannot be compacted. However, there is no visible folder called Temp.

I am hoping someone can help here - I have used Thunderbird since its inception and donate regularly to support it, but these issues are beyond me.

Roland

What is your anti-virus software? What is the size of your Inbox?

Hi

I appreciate you engaging with my question.

My anti-virus software is Kaspersky Total Security - I have had this installed for about five years, with regular updates. The issue with email moves or disappearance is a new issue.

My inbox is about 3.2GB, with about 23,000 messages. The total size of my Thunderbird profile is about 16GB across maybe 50 or 60 folders. Again, this has been the case for some years and I did not encounter any issues until the most recent major update.

Best wishes Roland

Anti-virus software is most likely the culprit. These are some generic suggestions to avoid problems with anti-virus software.

Create an exception in your anti-virus software for the Thunderbird profile folder, so that the anti-virus real-time scanner will not scan it. https://support.mozilla.org/en-US/kb/profiles-where-thunderbird-stores-user-data#w_how-to-find-your-profile

Don't let your anti-virus software scan incoming and outgoing messages.

Don't let your anti-virus software scan attachments.

Don't let your anti-virus software intercept your secure connection to the server.

Remove any add-ons your anti-virus software may have installed in Thunderbird.

Keep it working. http://kb.mozillazine.org/Keep_it_working_-_Thunderbird

And last but not least, backup your Thunderbird profile on a regular basis. https://support.mozilla.org/kb/profiles-where-thunderbird-stores-user-data#w_backing-up-a-profile

Note, anti-virus software often causes corruption of Thunderbird mail files. In the worst case this can also cause data loss, i.e. message may already got lost, even though you still see the headers. So you'll have to repair affected folders to clear the corruption. You'd then have to restore lost messages from a backup created prior to the corruption.

Also note, an Inbox of 3.2GB is asking for trouble. Keep your Inbox as tidy as possible, as it is most susceptible to corruption.

jollygreengiant, did you resolve your issue?

No - my issue is not solved. The problem seems to have settled down again and not recurred since the two most recent minor updates, so I will just carry on, and hope whatever caused the problem has now been fixed.

I must admit I found the only response depressing. I had said this issue is a new one since the latest major update. Neither my file or mailbox management nor my anti-virus have changed. I am also not sure its wise to turn off email scanning :-)

I have been a Thunderbird user for 20 years, make regular financial contributions, but I have never found these forums offer much in the way of help. I guess that is the reality of open-source products. Thunderbird is still my best option!

christ1 offered you some very important information.

You will find that your Anti-Virus gets updates all the time and so does Thunderbird. You will also find that the AV will scan anything downloaded , uploaded and any file opened. This causes a problem when it comes to emails because the AV interrupts the process. This is a well known fact and causes no end of irritations. Whilst on the topic - if you use a cleanup software product like CCleaner, WiseCleaner or indeed some AV offer it as well - then you need to make sure Thunderbird profile folder exempt as they could remove required data such as session or passwords etc. It mean preferences may not get updated upon closing and it causes annoyances.

If you really want the Anti-Virs to check everything - Never allow any Anti-Virus to auto fix anything to do with Thunderbird because AV's do not understand that an mbox file contains many (possibly hundreds) of emails. It sees a problem in a file and the auto fix on Inbox can be catastrophic. Loss of all emails.

Everytime a file gets opened in order to read or write data then that Anti-virus could be interupting the process to check an opened file. In addition, if you are using an imap account then there is a lot of jibber jabber between thunderbird and server for each email.

You may not have experienced an issue so far with Kaspersky, but I've been in contact with many people who have had issues. You would also find many people saying they have issues on Kaspersky help forum.

Emails themselves are not a problem - you can open email without an issue - it's what the recipient does with them which is important.

Basic safety rules : 1. Thunderbird does not allow remote content by default - so if you do not alter the default settings then you are safeguarded against remote content. 2. Do not click on links whether text or image before you know where that link is taking you to - hover over it first and the real link address/webpage is shown the bottom Status Bar. 3. Do not open email attachments just because you are curious.

I've had loads of emails claiming stuff like see you invoice attached - everyone should know if they have engaged with a company etc and are expecting an invoice.

re :My inbox is about 3.2GB, with about 23,000 messages. In reality that means if you download full copies of emails then in one single mbox text file document you have 23,000 emails all written one after the other. AS you seem to be very cautious then I would advise that you reduce your risk by creating more imap folders, then moving emails into those folders to improve organisation and storage. Remember to compact the Inbox after you have moved a lot of emails.

re : but I have never found these forums offer much in the way of help I notice you have not asked many questions in the past. But the information you are getting in this question alone is worth it's weight.

re :My inbox is about 3.2GB Assuming you are using Message Storage Type: File per folder (mbox)- this means you are talking about a single text file document which is 3.2GB in size - it's not a folder - it's a file. It may take quite a bit of RAM to open a single file of that size. Compacting becomes much harder if you have very large mbox files. Basically, any email you delete or move out of the Inbox, gets 'marked as deleted' and hidden. You may see the email in another folder or in Trash, but the original email is still lurking in the Inbox. When you compact the Inbox, the Inbox file is opened and read - all emails that do not have a 'marked as deleted' flag are copied over to a new file. Then the original mbox file is deleted and the new file replaces it. So the new Inbox mbox file is cleaner and smaller in size.

If mbox files get too big then in the event of needing to access it; maybe to recover data, can prove difficult. I have known computers crash when attempting to open very large files.

re :Example 1 - I moved an email from my Inbox to a special folder. The email was present in full before the move. Only the header appeared to be copied.

What do you mean by a 'special' folder ? Is that 'special' folder in another account eg: Local Folders mail account ? Are you saying you were 'moving' an email out of an imap folder - out of imap account and putting it in a folder you created in the 'Local Folders' mail account ? But Only the header was copied.

It might be only header copied over if only header was downloaded in the first place. Note: If only header downloaded - when you select to read email - contact is made with server and full copy gets downloaded to a temp cache to faciliate reading. So from your point of view, you would be able to read it anyway. If only header was copied/moved to 'Local Folders' folder then it cannot access server to download full copy. The only other explanation is a scanning process interrupted the process - already discussed in some detail.

As I'm not ware of all your specific settings on a per folder basis, I've included the following info as a reference guide. In Imap account :

• Account Settings > Synchronisation & Storage'
• Click on 'Advanced' button

Any folder which has a selected checkbox is designed to download full copies to that folder. Any folder which is not selected does not get a full downloaded copy - only a header.

In addition - if you have selected eg: 'Synchronise the most recent 60 days' option, then when you compact, only the last 60 days will be fully downloaded and all the rest will be just headers. This only works providing you have already selected the account folder to have full downloaded copies'

Normally compacting takes a few seconds and therefore does not really interrupt anything. You really need to see if you can reduce the current file size as it's not doing you any favours.

AS you have a large Inbox Compacting a large file may take a while. So advise you stop the AV from Scanning any file you open first.

Do not exit Thunderbird whilst the compacting process is operational. Strongly advise you compact on a per folder basis because you have large files. Advise you compact the folder that get used the most: Empty Spam and then compact Spam Then compact the 'Drafts' folder Finally try compacting the 'Inbox' If the 'Inbox' is being compacted then advise you do not select anything in that folder. In your case, it may be beneficial to go into Offline mode to stop the server from trying to download anything into the Inbox. Then try compacting Inbox. Maybe take the opportuity to have a cuppa.

Hi

First up, I appreciate you taking the time to reply in detail. My understanding is that you and others are volunteesr seeking to help others and I appreciate you giving your time to this.

Second, thank you for setting out your suggestions re compacting in your second message. While none of this was new to me, it was a useful reminder.

Thirdly, what comes through strongly is an opinion that the problems I am raising are either related to another program (Anti-Virus), or reflect some form of user error or lack of understanding or good practice on my part. I apologise if I may have contributed to this - for example "special folder" is old language, and I really should have said "user-created" - twenty plus years ago people talked about special folders and standard folders in email ;-)

I have been using Thunderbird since its inception and have a well-managed folder structure. The Inbox size is relative to the total size of this Profile, which is around 130GB.

For me the information about anti-virus is interesting, useful and important. It is also less likely to be related to the issues I was dealing with. This is simply applying Occam's razor.

My situation arose specifically after the large recent update and rebuild; it seems not to have recurred after the second minor update to this. This suggests the problem was related to some internal Thunderbird issues.

There are still likely to be issues. I attached a screenshot showing how the Search screen does not update fully after a move operation - in that it still shows the shading and "slots" for messages that have been moved. To me this is potentially related to the problem I experienced, and again is a new situation post the update.

Please ignore the "moving two items to Trash" at the bottom. This was subsequent to the move operation I reference above My point is that the slots on the screen are still showing as highlighted in a fixed position, even after that operation completed and I then moved to delete two items. At this point I realised I actually had tangible evidence of a problem and so took a screenshot.

I'd be interested in whether this is sufficient evidence to raise a bug report?

What I am left with in any event is needing to search my profile to discover whether any of the 200 odd missing messages from mid-July to early August are somewhere or are lost for good. The "Search" function simply fails when I look for a specific email address across all Folders in my local folder. It doesn't crash - it just stops without generating a result (ie it stops without finding anything or saying nothing found). It works across specific folders but not across all.

So I wonder if you are aware of any tool or other mechanism which one can use to directly search a large profile, outside of starting Thunderbird? As I understand it, messages are all text / HTML so it ought to be possible to search for a string of characters. Using the Windows 10 search function on a Thunderbird profile doesn't seem to work (I have tested this using known results).

Best wishes Roland

The following only works providing you have not compacted anything.

If you are using an IMAP mail account then you need to see if emails are on the server. If not on server, then Thunderbird cannot display them.

The image looks like you are talking about a load of emails in the 'Local Folders' mail account.

In Thunderbird

• 'Help' > 'Troubleshooting Information'
• click on 'open folder'

a new window opens showing profile folder name

• Close Thunderbird now - this is important

• If this was a pop account - click on 'Mail' folder
• click on pop mail account name or 'Local Folders' - whatever has missing emails.

Additional info that might be relevant' for imap accounts.

• If this was an imap account - click on 'ImapMail' folder
• click on imap mail account name that has missing emails.

you should see the 'Inbox' mbox file - it has no extension. I'm not sure what folder the deleted/lost message was originally in, but there will be an mbox file for that folder name.

I notice the search image mentions eg: Inbox/2017 So if '2017' is a folder you created as subfolder to 'Inbox' In 'Local Folders' folder - click on 'Inbox.sbd' folder to see mbox file called '2017' - then open that '2017' mbox file using Notepad ++

If the file is too big for Notepad++ then you need to find a different Text Editor program which can open a very large single text document)

Emails are written to the file one after the other, so oldest will be at the top and newest at the bottom. Each email will start with these lines: this is an example

• From - Sun Dec 28 18:14:40 2014
• X-Mozilla-Status: 0001
• X-Mozilla-Status2: 00800000

• Use 'Edit' > 'Find'

Try to locate the deleted email, by searching for the Subject or part of the email address etc. OR just search for 'X-Mozilla-Status:' and edit the number.

When you have located the email: Look for this line: X-Mozilla-Status: (not X-Mozilla-Status2:) Typically emails that have been moved/deleted will have these numbers : 0019 or 0009 So as you work down through file - if you see emails with those number - edit them.

• Make sure the X-Mozilla-Status: has the number 0001
• Save the file.
• delete the 'Inbox.msf' file. A new one will be auto created.

The above works ok for pop accounts and the 'Local Folders' mail account.

IF you are talking about an Imap mail account then it's likely emails do not exist on server so you need to perform an additional action. Do this if the file was in an imap account.

• Copy the saved mbox file.
• Paste the saved file into the 'Mail' >'Local Folders' mail account folder.
• Then rename file eg: rename as 'OldInbox' if necessary.

Restart Thunderbird.

Additional l images that may help with previous comment I posted.