It is not NVIDIA only, though. NVIDIA is just one of those factors that make the issue more frequent. Personally, I am experiencing the issue with Intel and AMD graphics. The machine I have on which the issue is more frequent at all is Chuwi two-in-one with a weak CPU (N4120), Intel graphics and EMMC/SD-card storage.
There is a piece of bad news, as one use is reporting that the issue is still present with the latest 6.11RC that should have a fix. Apparently, as some systemd developer suggested, there may be more pieces to fix before the freezing of user sessions is working with no hiccup.