Oof, after searching the web some more I found out that there is a hidden ff setting (about:config) for this:
widget.disable-workspace-management
Set it to true and enable preferences > general > restore previous session to have firefox start on the current virtual desktop and still restore tabs.
reference: