Your mount procedure is wrong.
You can’t mount the /dev/mapper/xyz device directly - mount the BTRFS subvolumes on it in the correct order instead.
A little further down in this post you find the correct procedure for a BTRFS file system.
Instead of /dev/sdaX in the example use your decrypted /dev/mapper/xyz
My experience with BTRFS is non existent - can’t help there.