AMD drivers are built into the kernel and as you’re running a non-LTS kernel, you will every now and then run into issues like this, so install both 5.4 and 5.10 LTS and stay with those if it works flawlessly there.
For more information as to the why, please read this: