Performance differences between xfce and KDE will be small. If you want a lean system you could also consider the minimal edition of KDE, won’t help in performance but in system size. For the performance the kind of your disk might make a difference especially for booting a SSD is a huge improvement compared to a HDD. Customizing in KDE is superior, Gnome is rather on the opposite.