I used live usb and removed all drivers and installed free drivers, now I’m able to boot normally.
I also feel dumb that I ignored this post here before: [Support/How-To] Compile old/legacy Nvidia Proprietary Drivers [2020-12-30 Update Issue]
I used it to compile 455 drivers and now things are working fine. Sorry again for the spam.