This is not a matter of Alacritty, it has to do with colorful output in pacman and yay, which is turned off by default.
You have to enable (uncomment) the option Color in your /etc/pacman.conf file. This will enable colorful output in both pacman and yay.
It’s documented in yay's GitHub repo here: