Start with Command-Option-Escape to bring up the Force Quit dialog and see if it's there. If it is, force quit it (sounds like you've already tried this).

   Open Terminal, and run ps -eaf | grep Firefox. If Firefox appears in the list (note that there will always be an entry for grep Firefox, this is not what you want, the actual Firefox will be listed as something like /Applications/Firefox.app/Contents/MacOS/firefox), note the PID (the second column, a number), and run kill -9 <PID>.

If neither of these work, try restarting the Dock (maybe Firefox actually isn't running but the Dock is confused):

   Use the same ps -eaf | grep Dock.
   There should be a process called /System/Library/CoreServices/Dock.app/Contents/MacOS/Dock, note the PID and run kill <PID>.
   The Dock should disappear momentarily and then restart.

If all this fails, easiest thing to do is probably just reboot your machine. If the usual shutdown dialog fails, press and hold the power button until the computer powers down.

Well shame on myself. The turn it off turn it back on answer worked. Sometimes forget to try that with OSx