First of all, you need to understand the mechanics behind grub
. A lot here depends on whether you boot up in legacy BIOS mode or legacy BIOS emulation mode (CSM) on the one hand, versus native UEFI mode on the other hand.
In the first case, the boot.img
component of grub
is installed into the master boot record, immediately followed by the core.img
component.
This works well on — and was designed for — a drive partitoned with an MS-DOS-style MBR partition table, but if the drive was partitioned as GPT, then there isn’t enough space between the master boot record and the first partition.
Therefore, booting grub
in legacy BIOS mode on a drive with a GPT partition table requires that one would create an empty, unformatted partition of type bios_grub
, of about 2 MiB in size, and marked with the boot
flag. It doesn’t matter where the partition is created, as long as it’s on the same drive and that it’s of the correct type, with the boot
flag set.
This partition will then be reserved for the core.img
component of grub
, and if you have not created this partition, core.img
will be written over the starting boundary of the first partition, thereby damaging the filesystem on the first partition — this has nothing to do with btrfs
, because it would do the same if your partition was formatted with ext4
or any other filesystem type.
On machines that boot in native UEFI mode, a different version of grub
is used — one that runs in the 64-bit long mode of the boot processor, as opposed to the legacy BIOS version, which runs in 16-bit “real mode”. The idea here is that one does not install grub
into the MBR — which is exactly what you did — but into a designated EFI system partition, formatted with a FAT32 filesystem and about 300 MiB in size.
As if that’s not complicated enough, when using btrfs
with the default layout — whereby /boot
is not a separate volume but rather just part of the root filesystem — you need to use a special version of grub
, namely grub-btrfs
. This is because btrfs
uses compression and sparse files, and with /boot
on the btrfs
root filesystem, your kernel images and initcpio
s are on btrfs
as well, which the regular grub
cannot handle because it does not support sparse files.
The proper command for (re-) installing grub
in a system with a GPT partition table and booting in native UEFI mode is simply…
sudo grub-install
… when doing so from within the installed system itself, in which case the command will pick up all of the correct defaults from the already installed system. If on the other hand you’re going to be explicit about where to install it, then you need to give it the EFI system partition as the target, not the MBR — /dev/sdb
is the drive itself, starting with the MBR.
So, indeed, by running “sudo grub-install /dev/sdb
”, you have damaged your btrfs
filesystem, and then of course, that damage cannot be repaired, even if only because your partition boundary is gone as well. The partition would need to be recreated and reformatted in order to fix that.