I figured out how to fix it
Edit /usr/share/pulseaudio/alsa-mixer/paths/analog-output.conf.common and add Element Master above Element PCM and it should look like this.
[Element Master]
switch = mute
volume = ignore
[Element PCM]
switch = mute
volume = merge
override-map.1 = all
override-map.2 = all-left,all-right
https://askubuntu.com/questions/1119938/audio-volume-doesnt-change