As it just so happens to be, I have had a couple of serious issues myself with Plasma only a few days ago, and it took me until yesterday to — with the help of several forum members — hunt down the cause and remedy the problems.
If you’re having stability problems with Plasma itself — and as you can read on the thread linked above, it does unfortunately occasionally happen — then the cause is most likely something in your home directory.
In many cases, it’s the use of an old and buggy theme that’s no longer compatible with the recent Plasma versions. In my case, it was most likely caused by a couple of corrupted configuration files as the result of a glitch while changing a few settings, and yes, even one single corrupted configuration file can do this, because the way Plasma is written, it is very sensitive to that.
Either way, the problem is most likely caused by something in your home directory — a corrupted configuration file or database, a broken theme, et al — and thus, it can be fixed, even if it means having to delete your Plasma configuration files and starting all over again from the template in /etc/skel.
Still, if you would prefer to stick with XFCE, then that is of course your prerogative. Overall, Plasma is pretty good, but its occasional quirks have driven people away from it before, and XFCE is a pretty robust environment with a decent reputation, even though it’s not my personal style. 