Pulseaudio stuttering/back-skipping when volume adjust

Hello. I installed a new soundcard (Xonar D1) and now I get very bad stuttering/back-skipping playback whenever I adjust the volume.

Device-1: Intel 7 Series/C216 Family High Definition Audio driver: snd_hda_intel 
Device-2: C-Media CMI8788 [Oxygen HD Audio] driver: snd_virtuoso 
Sound Server: ALSA v: k4.19.167-1-MANJARO

Tried following the arch wiki glitches but changing

/etc/pulse/default.pa
load-module module-udev-detect tsched=0

did not solve the problem.

Output from pulseaudio -v after a single volume adjustment:
I: [pulseaudio] client.c: Created 81 "Native client (UNIX socket client)"
I: [pulseaudio] protocol-native.c: Got credentials: uid=1000 gid=1000 success=1
I: [pulseaudio] client.c: Freed 81 "pactl"
I: [pulseaudio] protocol-native.c: Connection died.
I: [pulseaudio] client.c: Created 82 "Native client (UNIX socket client)"
I: [pulseaudio] protocol-native.c: Got credentials: uid=1000 gid=1000 success=1
I: [pulseaudio] module-device-restore.c: Storing volume/mute for device+port sink:alsa_output.pci-0000_05_00.0.analog-surround-51:analog-output.
I: [pulseaudio] client.c: Freed 82 "pactl"
I: [pulseaudio] protocol-native.c: Connection died.
I: [pulseaudio] client.c: Created 83 "Native client (UNIX socket client)"
I: [pulseaudio] protocol-native.c: Got credentials: uid=1000 gid=1000 success=1
I: [pulseaudio] client.c: Freed 83 "pactl"
I: [pulseaudio] protocol-native.c: Connection died.

I was also getting a timeout error but did not manage to reproduce right now.
Adjusting volume in alsamixer has no problem so I suppose it’s from pulseaudio.
No problems from network sound server or onboard sound (at least don’t seem related, that’s a whole other topic).

Discussing the problem here: Volume adjustment causes output stuttering/rewind