The minimum file size of the places.sqlite file is 10 MB and Firefox increases this file in 10 MB chunks when necessary (its size is never reduced when deleting history or bookmarks). This is done for performance issues caused by fragmentation.

It sounds that the current copy of places.sqlite file is corrupted and in such a case it is best to delete the places.sqlite and restore the bookmarks from a JSON backup.

• http://kb.mozillazine.org/Bookmarks_history_and_toolbar_buttons_not_working_-_Firefox
• https://support.mozilla.org/kb/Bookmarks+not+saved#w_fix-the-bookmarks-file

Hi Thank you, but I really am not interested in the Bookmarks, all I need is the browsing history.Because this is what causes the visited links to change color. All the questions that I have finished should stand out so I will be able to keep track of my progress.

As soon as my places.sqlite file reaches 30,720 KB ( or that is at least what I think) visited links or sites do not change their font color.

Try installing and running the addon places maintenance that may fix problems with your bookmarks database. It also has an option to generate a short status report. Please paste the status report into your next message (or attach a screenshot).

• https://addons.mozilla.org/en-US/firefox/addon/places-maintenance/

Thanks John,

I had already tried it before, but tried this add on once again. No joy. I have attached the Screen shot

Really appreciate everyone's help. I really do not want to loose my history. All those questions that I have visited over time have a different color then the ones not visited. So if I loose my history all questions will have the same font, i wouldn't be able to differentiate among done and non done ones

Will loose all of my hard work

I am sure with expert support here; we will be able to find a solution

Although places maintenance says data base is corrupt, do you think if changes are made to any of the following parameters in about:config,then it may help.

browser.history.maxStateObjectSize;655360 browser.sessionhistory.max_entries;50 browser.sessionhistory.max_total_viewers;-1 places.history.expiration.transient_current_max_pages;25913 services.sync.log.logger.engine.history;Debug

Eagerly waiting for your any next suggestions please.

Thank you,

Unfortunately I cannot think of any good solution.

First of all find and backup the file places.sqlite , it is in your profile, and is the database that holds the History. That at least preserves the current state. Find it from the troubleshooting option of Firefox, and put it somewhere safe such as the desktop. If you need to use that again then use a copy only.

• Profiles - Where Firefox stores your bookmarks, passwords and other user data

Not sure I can think of an addon that does what you require and tracks viewed and none-viewed-links (of course Firefox does! ) but you could look through the addons & see if anything suitable exists.I suppose one thing you could do is bookmark all links you have seen, or if the course is in chronological or numerical order maybe bookmark and you have seen nearly all the links bookmark the ones you have not yet seen.

It appears from the screens shot that the database is corrupt, If it does have to be replaced then try any option places-sqlite offers, there is a slim chance the History will be retained. After the repair/replacement run the status report again.

• What was showing off screen on the image ?
  • edit Try pasting the messages log from the places maintenance into your next message. I have just used the addon myself so can confirm it is pasteable text.

As long as you have backed up places sqlite you may at least create a new additional profile with the old places.sqlite solely for accessing pages with the present history status. (just do not change anything else or click additional links just copy them) It is even possible to run two instances of firefox.

Thanks John, The complete message as you requested is pasted below.

> Integrity check - The database is corrupt > Reindex - Unable to reindex database > Integrity check - The database is corrupt - Unable to fix corruption, database will be replaced on next startup

It seems that my backed up copy of places.sqlite may be corrupt as every time I create a new profile and replace the new places.sqlite with the old places.sqlite, the same thing happens.

Dont know why firefox sync is not helping in this matter.

My idea was to upload the present history status into my sync account then create a new profile and sync the new Firefox with my sync account , so that should download my previously uploaded history . It should be as though I am using a different computer.

That is why they have sync don't they, so that we can get our history and bookmarks on the different computer?

So from the first computer I upload the History data in the firefox servers and then I travel home, at home I open a different computer and I log in into fIrefox with the same account that i used for uploading , and then I download the history that I had previously uploaded.

But even after syncing many times my old history does not get downloaded. I guess a corrupt data base does not get uploaded in the first instance.

Can you tell me what other files in my profile store the history e.g. places.sqlite-wal places.sqlite-shm localstore.rdf etc

For now, from chrome I have managed to import Firefox history, and I am continuing in chrome from here. I have learnt to take Chrome History back up. So now hopefully everything should be fine in Chrome.Visited links are changing color as expected. So I can keep track of the ones done and not done.

But I will not give up on Firefox, I do have the history status as of today . Thanks for your help; At least I know where to come in case I find a solution.

Let me know if you need any thing else in order to better diagnose the

issue 

Important: Can you tell me exactly how a visited link changes color? Its obviously a code, in which file is this code, is it in places.sqlite. How is it related to history?

Thanks

Sorry it is not looking good.

1. The History unlike bookmarks is not backed up elsewhere by Firefox.
2. There is one possibility using Windows.
3. Sync is not intended to back things up to the Sync server
4. If places.sqlite does not sort itself out you may need to delete it to recreate. You will then loose your history, but can still access it indirectly IF you backed up a suitable copy
5. To try different databases create new profiles

1. History
   is rather less robust than bookmarks,you would have needed to deliberately somehow back it up, but that is too late now
2. Windows Previous Versions
   This is really the last chance. Make certain you have whatever places.sqlite you have had backed up. Backup always before and again after each and any restore attempt. Windows should be able to make a previous version available. One of those versions may be usable as is; or usable after a repair with places maintenance. Obviously that is not the most recent version but it should help.
3. Sync is not a server backup service.
   Sync is intended to backup to a second device, any backup is the fact that the second device then contains synced information. IIRC Sync development has ceased, and the replacement when available may include a server backup option.
4. Recreating Bookmarks Database (This includes History)
   Unfortunately the recreation can only use backed up bookmarks, there is no backed up History to use. Once the data base is recreated it will continue to store history but it loses the existing History. This time take steps to back it up on a regular basis.
   • Backup the profile or at least the places.sqlite within that
     Back up and restore information in Firefox profiles
     https://addons.mozilla.org/en-us/firefox/addon/febe/
   • http://kb.mozillazine.org/Bookmarks_history_and_toolbar_buttons_not_working_-_Firefox#Rebuild_Places_database
5. Use new profiles for database testing.
   To simplify comparing the results from the differing database files create one or more additional new profiles. You may then try overwriting the places.sqlite file in the test profile with any other places.sqlite you wish to try.That allows you to directly compare the databases without trying to use sync, and without changing the working Firefox profile unnecessarily. You can retain one profile for studies reference with the best of whichever profiles shows some history results; with whichever the best recreated or corrupt database is.
   • Profile Manager - Create, remove or switch Firefox profiles
   • CARE, do not rename or delete profiles once created, or use anything other than empty folders when creating them. Do not nest within other profiles. (At least until you know exactly how this behaves - post back if you are ever considering such actions)

If places.sqlite is corrupt then your only option is to delete this file and let Firefox create a new file.

Note that you can move/copy history items to a bookmarks folder and may be able to export the bookmarks including that copied history if that part of the library is still functioning.

Thanks John,

I have some very good news to share, but will share that in the next post after I have heard from you regarding the following queries

if i wanted to take the profile folder back up where would I find the folder ?

I can see my profile folder at 2 places 1) c:/users/admin/appdata/local/mozilla/firefox/profiles 2)c:/users/admin/appdata/roaming/mozilla/firefox/profiles

now both of these profiles seem to be same, so which one should I back up, or can I back up any of these two.

Secondly is it better to back up only places.sqlite or the entire profile.

Cant wait to share the good news, waiting patiently for your response. Thanks

The main profile folder is in AppData\Roaming, the location in AppData\Local is for the disk cache and other temporary files.

• C:\Users\<user>\AppData\Roaming\Mozilla\Firefox\Profiles\<profile>\

• http://kb.mozillazine.org/Profile_folder_-_Firefox

If you think the profile is a good one and your disk or other storage space is not critically low you may as well backup the whole profile.

That will include some more settings the most important ones probably being any passwords you have saved.

If you want to limit the size of places.sqlite, you can block favicons and site icons. Those things really bloat the bookmark storage. They are not saved in the .json files, so to get rid of them, you could dump the bookmarks and then restore them from a .json file.

Now the good news!

Well without the persistent encouragement which I received from both John and cor-el I would probably have given up on Firefox. John especially your support kept me going.

It was John’s last post containing the 10 vital points that lit the bulb in my head.

I realized that even after creating different profiles and pasting by old places.sqlite in the new profile, it wasn’t working.

So some how I had corrupted my backed up places.sqlite file. I usually back up my places.sqlite on a regular basis, replacing the old one with the new one at the end of each day Only this time, places.sqlite might have got corrupted at the end of the day , and without me realizing it was corrupt, I backed it up as usual.

Replacing the previous good version.

So now the good copy of places.sqlite was backed up by the corrupted one. hence the backed up copy was not helping.

Now thankfully I was able to find another copy of places.sqlite at a separate place that was not the latest one , its size was 20,520 KB .

So I created a new profile then replaced the places.sqlite in the new profile with the 20,520 KB places.sqlite.

Still I was not happy as about 10,000KB of history was still lost.

Then I tried to import all browsing history from Chrome, and guess what!

All browsing history is back in the non corrupted brand new data base. Earlier I had mentioned that from chrome I was able to import browsing history from firefox , even though the data base of firefox was corrupt, chrome was successful in importing all of my firefox's history, so later in the new profile , all I had to do was reverse the process.

Now simply I imported all of my chromes history back into firefox. With heart pounding, I ran the places maintenance to check the status of the data base after the import from chrome. Thank fully the data base was good! Below I have pasted the results of the test, and size is back to 30,720!

• > Integrity check
• + The database is sane
• > Reindex
• + The database has been reindexed
• > Orphans expiration
• + Database cleaned up
• > Coherence check
• + The database is coherent
• > Vacuum
• Initial database size is 30720 KiB
• + The database has been vacuumed
• Final database size is 30720 KiB
• > Statistics
• Database size is 30720 KiB
• user_version is 23
• page_size is 32768
• cache_size is -2048
• journal_mode is wal
• synchronous is 1
• History can store a maximum of 25913 unique pages
• Table moz_places has 29026 records
• Table moz_historyvisits has 48134 records
• Table moz_inputhistory has 5 records
• Table moz_hosts has 1996 records
• Table moz_bookmarks has 19743 records
• Table moz_bookmarks_roots has 5 records
• Table moz_keywords has 0 records
• Table sqlite_sequence has 0 records
• Table moz_favicons has 1232 records
• Table moz_anno_attributes has 13 records
• Table moz_annos has 675 records
• Table moz_items_annos has 6536 records
• Table sqlite_stat1 has 15 records
• Index sqlite_autoindex_moz_inputhistory_1
• Index sqlite_autoindex_moz_hosts_1
• Index sqlite_autoindex_moz_bookmarks_roots_1
• Index sqlite_autoindex_moz_keywords_1
• Index sqlite_autoindex_moz_favicons_1
• Index sqlite_autoindex_moz_anno_attributes_1
• Index moz_places_faviconindex
• Index moz_places_hostindex
• Index moz_places_visitcount
• Index moz_places_frecencyindex
• Index moz_places_lastvisitdateindex
• Index moz_historyvisits_placedateindex
• Index moz_historyvisits_fromindex
• Index moz_historyvisits_dateindex
• Index moz_bookmarks_itemindex
• Index moz_bookmarks_parentindex
• Index moz_bookmarks_itemlastmodifiedindex
• Index moz_places_url_uniqueindex
• Index moz_places_guid_uniqueindex
• Index moz_bookmarks_guid_uniqueindex
• Index moz_annos_placeattributeindex
• Index moz_items_annos_itemattributeindex

Well before I tried system restore, I gave this a shot and it worked. Without the encouragement provided by the support team, could not have had the motivation to carry on. So highly thankful to the support team.

I had more than 2 years of hard work that was at stake.

Now before I start celebrating, I read something in the places maintenance result that has me worried, I have italicized ‘‘History can store a maximum of 25913 unique pages

Now understandably next logical question, how do I increase this size?

Because after I reach 25913 unique pages no more pages will be stored in history,right? or what does "History can store a maximum of 25913 unique pages" mean. Is it something I need to be concerned about, never thought that there would be limits on the history? How can I increase this size, do not want re occurrence of the same problems again.

I cannot thank you people enough!Thank you

Easiest method will be to setup a separate profile for use only with your course. Delete all unecessay bookmarks or history in that profile.

That should leave you with two profiles that each have spare capacity in places.sqlite one profile for day-to-day browsing one for your course that needs a very large number of History entries. (Probably best to bookmark each one also)

places.sqlite deletes History once the database fills up. It gives priority to bookmarks. The maximum size was at one time settable manually. Now it is based on system resources.

• http://blog.bonardo.net/2011/09/02/changes-to-places-memory-consumption-expiration-and-performances
• https://developer.mozilla.org/en-US/docs/Places_Expiration

P.S. this article may be of interest to some viewers of the thread, as it explains the Import of Bookmarks & History from Google Chrome.

• Import bookmarks and other data from Google Chrome

Great to know you succeeded with this. Good luck with the course.

Well John, I cant say I am very happy to hear that the data base size has a limit.

So is there a way I can calculate the maximum size of my places.sqlite file in KB so that I know how much space i have left, currently I am using 30,720KB already so how do I know if this is the maximum quota or not.

It could happen that I may not even realize it and pages could already be expiring from the initial days .

Is there a way I can tell what will be the maximum size of my places.sqlite?

Thank you

There is no maximum for the places.sqlite database and other SQLite database files like I wrote above.
All SQLite database file have fixed minimum sizes and if they run out of space they are automatically increased in size with a specific chunk size. For places.sqlite this is 10 MB for the minimum and for the chunk.

• Bug 581606 - Avoid sqlite fragmentation via SQLITE_FCNTL_CHUNK_SIZE

Hi Cor-el,
I have not looked into this in depth and I am sure you understand it better than me, but I did think that; may be only in rare cases; History would be expired i.e. lost from places.sqlite. and that there would be a maximum, system dependent, size of the places.sqlite database.

Presumably I have missed finding some critical bug or document.

• Bug 581606 - Avoid sqlite fragmentation via SQLITE_FCNTL_CHUNK_SIZE
  This was fixed back in 2010
• http://blog.bonardo.net/2011/09/02/changes-to-places-memory-consumption-expiration-and-performances
  Is a blog a year after Bug581606 including:

.... - some users, with a really huge (tens of thousands) amount of bookmarks, were constantly losing history due to automatic expiration trying to keep the database to a sane size, from now on history of the last 7 days will not be considered expirable. ....  

That lead me to assume that History could be lost

• Bug 674210 - Reduce places.sqlite cache size and reorganize history expiration around the new value
  • This was fixed 2011
    .Yes I realise the bug relates to the cache size.
    However I also see comments in that relating to the database, and to expiry of History.

Looking at the bug's opening statement you can see why I get the impression there was intended to be a maximum size for the database and that History may expire in some cases if places.sqlite hits the maximum.

... The idea is to keep the current system but changing the heuristics to be more in par with the real system specs:

- Calculate a target max database filesize, based on these data:
* disk space available
* memory available
* number of cores (mostly if > 1)
* have a minimum size to fallback (even in case of broken system apis)
* have a maximum size for high-end specced systems (like our building boxes with 16 or more gigs)
- Calculate a max memory cache size based on the current size of the database
* have a minimum size to fallback and preserve performances ...

P.S. I strugle with the basics of C and have no knowledge of C++ but I guess the code involved may be this

• http://mxr.mozilla.org/mozilla-central/source/toolkit/components/places/nsPlacesExpiration.js

At least the code has useful comments in it.

Well both of you, John and Cor-el are right I think, lets think of it logically if my hard drive is full then where is Firefox going to store my history?

Obviously to prevent Firefox or any other browser from crashing , developers must have thought of a way to keep the browser going even if there is no more place to store the history, it is then that I think pages will start expiring, to accommodate the newer history.

So until your hard drive is full the database just keeps growing in small chunks, I think just as cor-el mentioned.

So its good to have free space in your C drive or where ever your Firefox profile is installed so that the condition of expiration never comes .

I am assuming RAM doesn't decide the database size , but the Hard drive.