Can I configure sound output to change automatically when attaching and removing an HDMI connection?

the sound still outputs to the laptop speakers

That would be expected behaviour for a laptop where the default sink is set to use analog output to laptop speakers

Simplest way to get audio playback to change sinks automatically is to load an additional PulseAudio module - module-switch-on-connect
PulseAudio - Switch on connect | wiki.archlinux.org
But this module will switch to any audio device when plugged in and overrides user settings for defaults

Users may prefer to change the default sink for PulseAudio (or PipeWire) to digital output for monitor speakers
How do I permanently set the default audio device in manjaro xfce with pipewire

The default sink can also be configured per-user in ~/.config/pulse/default.pa
PulseAudio - default.pa | wiki.archlinux.org
Or ~/.config/pulse/client.conf

When I unplug the HDMI connection… the output selection is blank and I have to select the built in speakers manually

That is not expected behaviour. If the HDMI sink output is not available, audio should playback should switch to an available sink output (module-rescue-streams on PulseAudio)
PulseAudio/Troubleshooting#Fallback device is not respected - ArchWiki