If using Firefox, definitely checkout Settings > General or about:preferences. Scroll until you hit the Fonts section and click Advanced. It should look something like this.
Regarding fonts in the Font Manager. I also disabled a number of fonts. Font Manager updates a file ~/.config/fontconfig/conf.d/78-Reject.conf.
I don’t think deleting a font is a long-term solution. Could possibly come back if in an update… There are some threads on font reduction that suggest using NoExtract in pacman.conf. I actually did a combo of Font Manager and pacman.conf, especially for noto. I did the same with the man pages. I really like to search the system with locate, and the number of superfluous results was time consuming.