Both versions recognize my hardware but do not behave the same. The xfce version will not recognize the primary monitor, it insists on using the left-most monitor as the primary. The KDE version handles this properly, the problem there being speed, The xcfe version is about 3x faster than the KDE version.
I have not had these issues with either Ubuntu or Mint.
Try randr utility (cli tool) or arandr (UI for randr, but with less functionality than cli version). Seek for guides how to use it and apply it on xfce, and see if it helps.
If you use both for the same user, then a lot of issues and inconsistencies will be present. Most likely the XFCE did not saved properly the
~/.config/xfce4/xfconf/xfce-perchannel-xml/displays.xml
so you might want to delete it as it will create a new one next boot.
Only on a faulty KDE Plasma install/config that might happen …