Removing the package will remove the driver. You need another driver installed for graphics to work if the nvidia card is your only graphics adapter or else you’ll probably fall back to text mode (where you could reinstall the removed package from cache).