[HowTo] Create a recovery setup detached from the actual system

WORK in progress…
==> this tutorial is a draft and should be improved
==> Feel free to test, edit and improve the content :wink:

Difficulty: ★★★☆☆

This tutorial is inspired by this one: [HowTo] Booting manjaro iso directly with grub

Introduction

In most cases, when you see a black screen, it is recommended to start a Manjaro installation media aka “live session”. But it can be that there is no USB stick and you just want to continue working. This setup should solve the following problems:

  1. No flash drive is available.
  2. ISO Images or Squash Images of any kind can be booted.
  3. TODO Start an initramfs that detects and fixes common problems via chroot.

How it works

The idea is nothing new, but is also used for bootable USB sticks. There is one installation of Grub for (U)EFI and one BIOS/CSM, which are independent of each other.
In case the UEFI installation does not start, you can start the BIOS installation and thus start a repair routine.
Of course, you can choose only UEFI or only CSM, but here you would have both options.
Another partition will then hold ISO images, squash images or an initramfs image. This is used by Grub as root and then boots the respective medium that was entered in the Grub configuration file.
If you choose UEFI only or BIOS/CSM only, then the additional partition for the images can be omitted.

In any case, the Manjaro boot loader remains untouched and will boot without further ado. An exception would be a pure BIOS Setup, which several Bootloader per non removable disk are not possible and the case is not supported.

Of course, problems can usually be solved with the help of the built-in rescue mode and this is true in most cases, but the terminal jargon is not for everyone where this system is supposed to support.

  1. A new installation can be made without having to have a USB stick at hand.
  2. A CHROOT can be performed to solve problems, but at the same time one does not have to do without a graphical environment.
  3. What should be the ultimate goal is to boot a custom image that does not boot into the command line, but opens a graphical menu that opens the possibility to solve problems in a user-friendly way. It should do that in general:
    • Scan for problems
    • Make possible problems visible
    • Give the user the possibility to solve the problem with one click
    • Save the process for further investigation if it fails
    • Optional: open terminal for manual repair
    • Optional: backup tool (cloning, snapshots, rsync)

Requirements

  • The drive must have a GPT (GUID Partition Table)
  • Manjaro must boot in EFI mode and optionally CSM could be used also.

Preparation

We need 4 partitions

  1. Partition for the grub image
    • Size → 8MB
    • Filesystem → none/non-formatted/deleted
    • Name → grub_bios
    • Label → none
    • Flag → grub_bios
  2. Partition for EFI boot
    • Size → 50MB
    • Filesystem → FAT16/32
    • Name → EFI_RECOVER
    • Label → EFI_RECOVER
    • Flag → boot
  3. Partition for CSM/BIOS boot
    • Size → 50 MB
    • Filesystem → EXT2
    • Name → CSM_RECOVER
    • Label → CSM_RECOVER
    • Flag → legacy_boot
  4. Partition for the ISOs etc…
    • Size → 5GB (and/or more)
    • Filesystem → EXT2 (but it can be any filesystem which grub can read)
    • Name → ISO_RECOVER
    • Label → ISO_RECOVER
    • Flag → none

You should create this in the GUI of parted: gparted or kparted.

:notebook: Name → Partition Name | Label → Filesystem Label

You may ask: “Why ext2 and not ext4?”. Because the extra features in ext3/4 are not needed for this use case on such small partition. You can upgrade ext2 anytime to ext3 or ext4, but no downgrading.

Now we create mountpoints and add them to fstab:

sudo mkdir -pv /recovery/{efi,csm,iso}

File: /etc/fstab

# RECOVERY SETUP
LABEL=EFI_RECOVER	/recovery/efi   vfat    rw,relatime,fmask=0000,dmask=0000,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro,noauto,x-systemd.automount           0 2
LABEL=CSM_RECOVER	/recovery/csm   ext2    defaults,noatime,noauto,x-systemd.automount        0 2
LABEL=ISO_RECOVER	/recovery/iso	ext2	defaults,noatime,noauto,x-systemd.automount	0 2
sudo mount -v -L EFI_RECOVER
sudo mount -v -L CSM_RECOVER
sudo mount -v -L ISO_RECOVER

If everything worked as expected, then lsblk -f should display something similar to this:

$ lsblk -f
NAME   FSTYPE FSVER LABEL       UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
sda
├─sda1
├─sda2 vfat   FAT32 EFI_RECOVER E2BD-CDE3                              36,7M    25% /recovery/efi
├─sda3 ext2   1.0   CSM_RECOVER 5465bf5d-eaa1-4dd7-9b0d-9eef184edae9   31,9M    25% /recovery/csm
├─sda4 ext2   1.0   ISO_RECOVER 5b344de2-9ea4-45bc-8e70-694c1e5f48e7    3,2G    62% /recovery/iso  

Install grub for EFI and CSM

We need to install it separately.

EFI

sudo grub-install --target=x86_64-efi --recheck --removable \
--bootloader-id=Recovery \
--efi-directory=/recovery/efi \
--boot-directory=/recovery/efi/boot 

CSM

sudo grub-install --target=i386-pc --recheck --removable \
--boot-directory=/recovery/csm/boot \
/dev/sda

Change the permission

sudo chmod -Rv 777 /recovery/csm
sudo chmod -Rv 777 /recovery/iso

Create the Menu

Since these grub installations are independent, we need to create the grub.cfg by hand. grub-update will not work.

Create keyboard layout of your language

sudo grub-kbdcomp de --output /recovery/efi/boot/grub/de.gkb
sudo grub-kbdcomp de --output /recovery/csm/boot/grub/de.gkb

Replace de with your language. Available symbols are extracted from here: /usr/share/X11/xkb/symbols/

:notebook: Not all Symbols can be used, therefore it will reject symbols, which cannot be used in the layout file. It is limited ot ASCII, but the keys will be then on the correct place.

EFI

File: /recovery/efi/boot/grub/grub.cfg

insmod part_gpt
insmod ext2

# Load Keyboard Layout
insmod keylayouts
keymap /boot/grub/de.gkb
terminal_input at_keyboard

# Load EFI video drivers. This device is EFI so keep the
# video mode while booting the linux kernel.
insmod efi_gop
insmod font
if loadfont ${prefix}/fonts/unicode.pf2
then
        insmod gfxterm
        set gfxmode=auto
        set gfxpayload=keep
        terminal_output gfxterm
fi


menuentry "Manjaro ISO"  {
    set isofile="/manjaro.iso"
    set label="ISO_RECOVER"
    set dri="free"
    set lang="de_DE"
    set keytable="de"
    set timezone="Europe/Berlin"
    search --no-floppy --label --set=root $label
    set pqr="/dev/disk/by-label/$label"
    loopback loop $isofile
    linux  (loop)/boot/vmlinuz-x86_64  img_dev=$pqr img_loop=$isofile driver=$dri tz=$timezone lang=$lang keytable=$keytable
    initrd  (loop)/boot/intel_ucode.img (loop)/boot/amd_ucode.img (loop)/boot/initramfs-x86_64.img
}

CSM

File: /recovery/csm/boot/grub/grub.cfg

insmod part_gpt
insmod ext2

# Load Keyboard Layout
insmod keylayouts
keymap /boot/grub/de.gkb
terminal_input at_keyboard

menuentry "Manjaro ISO"  {
    set isofile="/manjaro.iso"
    set label="ISO_RECOVER"
    set dri="free"
    set lang="de_DE"
    set keytable="de"
    set timezone="Europe/Berlin"
    search --no-floppy --label --set=root $label
    set pqr="/dev/disk/by-label/$label"
    loopback loop $isofile
    linux  (loop)/boot/vmlinuz-x86_64  img_dev=$pqr img_loop=$isofile driver=$dri tz=$timezone lang=$lang keytable=$keytable
    initrd  (loop)/boot/intel_ucode.img (loop)/boot/amd_ucode.img (loop)/boot/initramfs-x86_64.img
}

Test it

When everything is set correctly, you can test it.

  1. Copy the manjaro.iso to /recovery/iso
  2. Look up the Hex Number of Recovery:
efibootmgr
  1. Set the number for the next boot:
sudo efibootmgr -n 0001
  1. Reboot now:
reboot

TODO

  1. At this point it is also possible to create an initramfs, which includes essential tools to manage broken installations automatically. That is one thing I looking to achieve.
  2. Add more example Menu entries.
  3. Add an introduction describing how the setup will work.
7 Likes

I think it’s good to work on an unbreakable manjaro

Because: When updating,

  • the kernel may be replaced
  • the modules need to be replaced
  • all initrds need to be replaced too

In this scenario there is a time window (and it is huge, several minutes) where:

  • all kernels are deleted
  • alle modules are unavailable
  • the initramdisks are deleted

When interrupted in this moment the next boot may be a desaster

@andreas85 I cannot follow you.

Yeah, on updating theses files/folders are replaced, but the initramfs itself contains everything what is needed. And maybe I was not clear enough: The initramfs will a custom one, which includes more files than the “default” or “fallback” one.

Maybe I misunderstood the concept. Are there any links to the modules of the actual root, when I place it on a different root?

Never mind. I agree with you. At the moment, I am working on a custom initramfs which will chroot to the actual system with init hook.