I can think of three different Pinterest "pin it" buttons:

(1) Button on Firefox's main toolbar. This is generated by an extension, and you can remove it from the Add-ons page, Extensions category.

(2) Button on Firefox's Bookmarks Toolbar. You should be able to right-click > Delete this button.

(3) Button in the web page, added by the website itself. This is the hard one. You probably need an add-on to block the page from loading or running the script that generates these buttons. Or which hides them.

But it's a little more complicated. I've integrated Pinterest buttons into some of my sites without thinking anyone would find them annoying. For example on this page, when you click an image, it shows near the caption:

http://www.jeffersonscher.com/photos/rest/ShaoMountain/

Since that is generated by my own page, even if you block Pinterest scripts, it would still appear. Hmm...

Could you give an example of a page where you find it annoying?

You may also have specific cookies that cause such buttons to show.

You can remove all data stored in Firefox from a specific domain via "Forget About This Site" in the right-click context menu of an history entry ("History > Show All History" or "View > Sidebar > History") or via the about:permissions page.

Using "Forget About This Site" will remove all data stored in Firefox from that domain like bookmarks, cookies, passwords, cache, history, and exceptions, so be cautious. If you have a password or other data from that domain that you do not want to lose then make sure to backup this data or make a note.

You can't recover from this 'forget' unless you have a backup of the involved files.

If you revisit a 'forgotten' website then data from that website will be saved once again.