I think you mean the toolbar, yes?

Customize Firefox controls, buttons and toolbars {web link}

Type about:customizing<enter> in the address bar.

Note: about:customizing is no longer present in Firefox 47+ releases.

• Press Alt or F10 to bring up the Menu Bar.
  Then View > Toolbars > Customize.
• Right-click on a blank spot in the toolbar and
  select Customize.

In the new window, look for the icon. When you find it, hold down the left button on it, and move it to where you want it to be. You can move any of the icons you want from here, but; some icons are locked in place.

There is also a Restore Defaults button at the bottom middle.

No, showing the name of an extension on cursor hover is the only way that the name will appear.

But there is an Overflow drop-down that does display the extension name in that overflow when you open it. But that is when the Toolbar button is removed from the Navigation Toolbar only and I doubt if that would be of interest to you.
See for yourself:
https://support.mozilla.org/en-US/kb/customize-firefox-controls-buttons-and-toolbars#w_customize-the-overflow-menu-or-the-toolbar

Intended to "clean up" the Toolbar for users who have too many extensions with Toolbar buttons, that crowd that area and reduce the size of of other elements that are "flexible" on the Navigation Toolbar. I use that "overflow" for extensions that I seldom use.