Install dconf-editor, go into /org/gnome/desktop/input-sources/xkb-options
The, you will find a box for “custom values”. I added ‘caps:swapescape’ and ‘compose:ralt’, separated by commas, inside the surrounding bracket (it already had ‘lv3:ralt_switch’)
You can achieve the same with the “gsettings” command.
Now it seems to stick.
Not sure why the other methods didn’t work. A bug maybe?