Has bookmark file compression changed in Firefox quantum?
In Linux Mint 18.1, Firefox 62.0.2_64, I installed (extracted) Fx to /opt, in its own new directory (/opt/firefox). Earlier, I created a new profile using profile manager, for an earlier Quantum version Fx 60.x esr..
Then copied several of MOST recent days of bookmark backups (.jsonlz4 files) from the PRE-Quantum profile into the new Quantum profile above, that has only been used with Fx Quantum versions. Note: Some of the jsonlz4 files were copy & pasted from old legacy to new quantum profile; after that, Quantum backups were automatically created (so should be compressed?).
Are there easier ways than I know about, to quickly tell if 2 bookmarks files contain *approximately* the same number of non-corrupted bookmarks - in a human readable way, or file comparison (in Linux) that creates a list of elements found only in one file?
Is there a way (say, convert to a spreadsheet or CSV, etc, to find all differences in specific column, or something similar?
I'm not so sure something didn't go wrong, moving bookmarks from a Legacy profile to Quantum profile.
The places.sqlite files are IDENTICAL size in both the old legacy profile vs. new Quantum profile. That's a little odd, as exporting to html - the most recent bookmark files of legacy & quantum profiles and comparing them, the legacy bookmarks.html files have a LOT more lines (~ 3000 more lines- "line numbers", in text a editor ). Graphics / icons count as one line, so larger icons or higher resolution doesn't explain a much larger # of LINES in the legacy html file . Maybe that's if bookmarks in places.sqlite don't contain descriptions - ever.
The automatically backed up Legacy bookmarks*.jsonlz4 are considerably larger than the equivalent Quantum automatic backups. The files I'm comparing should have nearly the same bookmarks. No significant additions or deletions of bookmarks were made to either Legacy or Quantum bookmarks, so that's probably not the size difference.
Mozilla said? they wouldn't SHOW bookmark descriptions starting in Fx 62 (that's now), BUT WOULDN'T REMOVE the descriptions DATA from bookmark files until a LATER Fx version, so Ghacks.net says.
Comparing the exported bookmarks from Legacy & Quanum profiles, seems clear the Quantum HTML files do not contain any descriptions. Quantum bookmarks*.jsonlz4 files are ~ *35% smaller than Legacy files (that should contain the same set of bookmarks.
I'd guess that either Mozilla removed descriptions sooner than promised, or moving Legacy bookmark files to Quantum profiles removes descriptions, intended or not.
All Replies (4)
The file name of each .jsonlz4 backup includes the item count (bookmarks + folders) so that is the first thing to check to assess how many are in there.
Firefox 62 still has description data; I'm puzzled that yours somehow got lost by simply copying files between folders. Did you open the exported HTML files in tabs and compare side-by-side or flipping back and forth?
In case it's helpful, I have a decompressor and converter here (dual hosted due to a recent issue):
"Firefox 62 still has description data" No... it doesn't. :) Not mine. It doesn't really matter if the bookmark files contain the descriptions data, if you can't see descriptions in the Fx 62 UI. It's not really doing any good.
I was more concerned about some bookmarks not transferring, due to a glitch, corrupt file, etc. There's now too much data showing they have (nearly) the same # folders + bookmarks, and bookmark descriptions are gone from Quantum profile.
No matter how I compared the exported html bookmarks from legacy & quantum libraries / profiles, it's hard to miss that the legacy file contains descriptions. That's why it's almost 3000 lines longer than the file from Quantum profile. The files are time stamped very near each other.
1st off - the exported legacy profile html file was 1.5 MB. The exported quantum profile html file was 925.3 KB.
I compared them side by side with a "side by side file comparison tool," - Meld (Linux). Also looked at them in text editor that colors different parts of file code (here, html).
Thanks about the tip on bookmark BU files showing the total # of files & folders. A backup file from quantum profile has almost the same total as the legacy file from the same day. I used both browsers, under different profiles, at different times for a couple of days, so it's not surprising one has a few more or less bookmarks.
The bookmark descriptions shown in the file from legacy profile but missing in the Quantum bookmark file are very hard to miss, side by side - or tab to tab, as the data present in one file but not the other is highlighted.
A exported bookmark in the html file exported from legacy profile:
[here starts descrip:]
Same bookmark in the html file from the quantum profile. Notice the add_date & last_modified dates are = in both bookmarks; no description:
<DT><A HREF="http://www.asus.com/Motherboards/Intel_Socket_775/P5EVM_HDMI/#specifications" ADD_DATE="1309977095" LAST_MODIFIED="1531807517">Asus P5EVM-HDMI_socket 775_G35 ICH9R</A>
Joebt said
"Firefox 62 still has description data" No... it doesn't. :) Not mine. It doesn't really matter if the bookmark files contain the descriptions data, if you can't see descriptions in the Fx 62 UI. It's not really doing any good.
That is a different point. Usefulness of the data is in the eye of the user. You are asserting that the data itself was deleted.
This was my test:
(1) Create a new Firefox 62 profile and restore a nightly .jsonlz4 backup from my regular Firefox 62 profile. (Steps at the end)
(2) Export to HTML format (bookmarks_restored.html) from the new profile. File is 1372 KB.
(3) Export to HTML format (bookmarks.html) from regular profile. File is 3921 KB.
(4) Open bookmarks.html in a tab and use Save As to text format (bookmarks.txt). This captures all the visible information in the page, plus URLs, and includes the descriptions. File is 764 KB.
(5) Open bookmarks_restored.html in a tab and use Save As to text format. This captures all the visible information in the page, plus URLs, and includes the descriptions. File is 764 KB.
Sorry for the typo:
(6) Use WinMerge to compare bookmarks.html and bookmarks_restore.html and they are identical.
(6) Use WinMerge to compare bookmarks.txt and bookmarks_restored.txt and they are identical.
Test Setup
(A) Create and Launch a New Profile
Inside Firefox, type or paste about:profiles in the address bar and press Enter/Return to load it.
Click the Create a New Profile button, then click Next. Assign a name like BmkTest, ignore the option to relocate the profile folder, and click the Finish button.
Windows:
After creating the profile, scroll down to it and click the Set as default profile button below that profile, then scroll back up and click the Restart normally button. (There are some other buttons, but please ignore them.)
Firefox should exit and then start up using the new profile, which will just look brand new. Please ignore any tabs enticing you to connect to a Sync account or to activate extensions found on your system so we can get a clean test.
Linux:
As a shortcut, you might be able to do this:
After creating the profile, scroll down to it and click the Launch profile in new browser button below that profile. Check the toolbar layout to confirm it's not your regular profile. If it is, use the method listed under Windows instead.
(B) Restore a backup
Use "Choose file" to access a backup file for comparison: Restore bookmarks from backup or move them to another computer.
(C) Returning to your default profile after Step #2
Windows:
When you are done with the experiment, open the about:profiles in the new profile, click the Set as default profile button for your normal profile, then click the Restart normally button to get back to it.
Linux:
If you didn't need to exit the regular profile, you can just close the window on the new profile.
Modified by jscher2000 - Support Volunteer
As a second test:
(1) Using my new Firefox 62 profile, I restored my last nightly .jsonlz4 backup from Firefox 52.9.0esr.
(2) Export to HTML format (bookmarks_52restored.html) from the new profile. File is 1289 KB.
(3) Export to HTML format (bookmarks_52esr.html) from 52.9.0esr. File is 1308 KB.
(4) Open bookmarks_52esr.html in a tab and use Save As to text format (bookmarks_52esr.txt). This captures all the visible information in the page, plus URLs, and includes the descriptions. File is 730 KB.
(5) Open bookmarks_52restored.html in a tab and use Save As to text format. This captures all the visible information in the page, plus URLs, and includes the descriptions. File is 730 KB.
(6) Use WinMerge to compare bookmarks_esr52.txt and bookmarks_52restored.txt and the changes between them relate to "smart bookmark folders" (place: URIs), no descriptions were lost.