Under the Option menu item click the Privacy and Security tab on the left. There you have options to have it not remember History, and for Cookies and Site Data choose Keep until: I close Firefox.

You can also use private browsing if you want to keep most of your cookies/history but want certain sessions to only be removed.

Type about:preferences#privacy<enter> in the address bar. The button next to History, select Use Custom Settings.

At the bottom of the page, turn on Clear History When Firefox Closes. At the far right, press the Settings button. Turn on Cache and Form And Search History and whatever else you want.