Sounds like this is a bug, if you can reproduce it on a new profile, or on another computer, please report it here: https://bugzilla.mozilla.org so the developers know and can fix it.

I bypass this by opening the link in a new tab/window. Once I am done, I close it and I'm back where I started.

Sounds like that is a webpage which loads very quickly with a small amount of data initially and as the user scrolls down the page additional content keeps being added as the user continues scrolling down the page. And before the user realizes that "additional pages are being added to the initial page load" there are a dozen or more pages appearing in the "Go back one page"cache.

And that is becoming more and more common as websites are being rewritten to the HTML5 standards to refresh websites for "mobile" devices. At least that is "my take" when I see that happening to websites that I frequent and see the webpage slider move upwards each time I get what "had been" indicated by that slider that I was nearing the end of that web page, when that didn't happen at the web site in the past. Overall, I find that unnerving nut once I realize what is happening I take that is progress towards modernizing that web site.

And as Fred mentioned, I usually open hyperlinks to content that I want to read in a new tab, so the "Go back one page" button or the contextual menu listing is something that I rarely use any more. I guess you gotta change with the times as the state of the art progresses, or new features are added to web page coding standards. And the "state of the art" with regards to tabbed browsing really changed about 15 years ago when Firefox started to "slay" IE and then 10 years ago when Google Chrome arrived on the scene.

Could you give a couple examples of sites that cause this? A few years back, there was a news site that added the next story title to the back button history as you browsed down through the stories. What's odd is that it didn't affect other browsers. Anyway, it's hard to submit a good bug report without knowing a page that reliably causes the problem.