Are you saying that changing a setting on 'remember' and 'never' will automatically go to custom? If so I think this is normal.

The "Use custom settings for history" choice allows to see the current history and cookie settings, but selecting this doesn't make any changes to history and cookie settings.
Firefox shows the "Use custom settings for history" selection as an indication that at least one of the history and cookie settings is not the default to make you aware that changes were made.
If all History settings are default then the custom settings are hidden and you see "Firefox will: (Never) Remember History".
"Never Remember History" means that Private Browsing is active and "Always use private browsing mode" gets a check mark.

"Use custom settings for history" stays selected if at least one of the History or Cookie settings is not the default to make you aware that changes from the default setting have been made.

I have been ticking off all the options one by one, and checking if the "remember all history" selection stays on or not. I found that the one option that causes this strange behaviour is the box:

"remember my navigation and download history"

Not ticking this box, and only this one, causes the "remember all history" option not to stay selected, and makes the pull-down menu go back to custom on its own. (I really can't belief that this is intended)

Ticking this box, and making any other changes, causes the functionality to work normal and as expected. Changing it back then to custom show you which boxes were then automatically selected by this option. Again, normal and expected behaviour in my opinion.

In reply to the answer above: I find this very odd behaviour to be honest. Such pull-down options are meant to quickly select a setting, not to be some sort of indicator. I would change that since it is confusing. Especially since choosing one of these pull-down options does cause boxes to be ticked on or off when you change the selection back to custom.

Edit: You can easily test this. Any combination of boxes ticked, with "remember my navigation and download history" also ticked, makes the "remember all history" selection work normally. So I can't belief this is working as intended. Why would you make this single box more important than any of the others??

and mine is just the reverse - Changing it to Custom will revert in seconds back to 'Remember history'.

Any relationship between these two behaviors. With the new 23.0.1 - I get a new tab labelled 'Please enable cookies', every time. Have tried everything I can think of, finally found this odd behavior on that Privacy option which WILL NOT persist.