If you haven’t changed your host system you will be on X11 and that should not cause problems with Virtualbox. See here how to check: [HowTo] Check a user session type (Wayland / X11) you are using on Gnome, KDE
Hard to find the issue with this, we probably can’t help much, could be Hardware, Virtualbox, everything. It sounds more like a slight program freeze of Virtualbox.
When running the VM, at the lower right corner there is an icon for mouse integration. With right click you can activate or deactivate it on the fly. Try that and see if it comes back. The host OS still has the mouse functioning.
Give enough hardware power to the VM, maybe reinstall Guest Additions and also check VM logs.