Can I configure sound output to change automatically when attaching and removing an HDMI connection?

When I plug in my TV via HDMI, the sound still outputs to the laptop speakers, and I have to go into settings to switch the output. When I unplug the HDMI connection, there is no sound output and when I go into settings, the output selection is blank and I have to select the built in speakers manually.

Is there a way to have the sound output switch automatically when I plug in and disconnect an HDMI device?

You could realize this via udev rule.

If you don’t want to change things system-wide (which requires super-user), you can have a script running as your own user that will react to device changes. I wrote one myself: auto_configure_upon_usb_device_change.py. I’ve been using it for years, across even distros. Regardless of how its called, it reacts to both USB, Bluetooth and display changes.

the sound still outputs to the laptop speakers

That would be expected behaviour for a laptop where the default sink is set to use analog output to laptop speakers

Simplest way to get audio playback to change sinks automatically is to load an additional PulseAudio module - module-switch-on-connect
PulseAudio - Switch on connect | wiki.archlinux.org
But this module will switch to any audio device when plugged in and overrides user settings for defaults

Users may prefer to change the default sink for PulseAudio (or PipeWire) to digital output for monitor speakers
How do I permanently set the default audio device in manjaro xfce with pipewire

The default sink can also be configured per-user in ~/.config/pulse/default.pa
PulseAudio - default.pa | wiki.archlinux.org
Or ~/.config/pulse/client.conf

When I unplug the HDMI connection… the output selection is blank and I have to select the built in speakers manually

That is not expected behaviour. If the HDMI sink output is not available, audio should playback should switch to an available sink output (module-rescue-streams on PulseAudio)
PulseAudio/Troubleshooting#Fallback device is not respected - ArchWiki