Audio stutters when KDE volume control widget panel is opened up

I am on testing branch with pipewire-pulse. Earlier I was using wireplumber but moved back to pipewire-media-session due to a bug that caused clementine to go silent after seeking.
Now I am experiencing this very weird bug that when I open up the volume control widget the audio stutters but only as long as the widget is open.
On top of that this bug interacts with the bug in clementine I mentioned above such that when volume widget is opened the audio comes back but with the usual stuttering.