This problem should have been fixed in Firefox 45!

To remove it, you can use either of these methods:

Options page

Switch over to the Options page, and select General in the left column. Toward the top of the page, in the Startup section, check the Home Page field:

Look for a vertical bar character. As a simple example with short URLs:

https://mozilla.org/|about:preferences

The bar definitely can be hard to spot with long URLs, but when you find it, delete whichever of these you find --

|about:preferences
|about:preferences#general

-- I suggest deleting the bar last so things don't accidentally get mushed together.

Firefox will save the edit when you tab out of or click away from that box.

In Firefox before version 45, don't click the "Use current pages button" or you'll have to start again. In Firefox 45, only the open tabs other than the Options page should be added.

about:config Preferences Editor

(1) In a new tab, type or paste about:config in the address bar and press Enter/Return. Click the button promising to be careful.

(2) In the search box above the list, type or paste home and pause while the list is filtered

(3) Double-click the browser.startup.homepage preference and remove whichever of these you see --

|about:preferences
|about:preferences#general

-- I suggest deleting the bar last so things don't accidentally get mushed together.

The about:preferences page gets possibly included as one of the home pages if you click "Use Current Page(s)" to set the home page and about:preferences is open in a tab. If you want to use the "Use Current Pages" button in "Options/Preferences > General" then you can pin the tab with the about:preferences page via the right-click context menu of the tab to prevent including about:preferences to the home page setting. You can drag the globe/padlock at the left end of the location/address bar to the toolbar Home button to set a single page as the home page.