Can not boot Manjaro after crash during update

Support
,
1

Hi there,

during updating via Manjaro GUI Package Manager, my laptop crashed, now I am not able to boot.

I was able to

  • boot from a Live usb stick
  • to mount the encrypted NVME and see my data
  • use manjaro-chroot
  • run grub-install and update-grub without errors

Here is the output of

inxi -Fxxxza

manjaro@manjaro ~]$ inxi -Fxxxza 
System:    Kernel: 5.10.23-1-MANJARO x86_64 bits: 64 compiler: gcc v: 10.2.0 
           parameters: BOOT_IMAGE=/boot/vmlinuz-x86_64 lang=en_US keytable=us tz=UTC misobasedir=manjaro 
           misolabel=MANJARO_XFCE_210 quiet systemd.show_status=1 apparmor=1 security=apparmor driver=free nouveau.modeset=1 
           i915.modeset=1 radeon.modeset=1 
           Desktop: Xfce 4.16.0 tk: Gtk 3.24.24 info: xfce4-panel wm: xfwm4 vt: 7 dm: LightDM 1.30.0 Distro: Manjaro Linux 
           base: Arch Linux 
Machine:   Type: Laptop System: LENOVO product: 82MS v: Yoga Slim 7 Pro 14ACH5 serial: <filter> Chassis: type: 10 
           v: Yoga Slim 7 Pro 14ACH5 serial: <filter> 
           Mobo: LENOVO model: LNVNB161216 v: SDK0R32862WIN serial: <filter> UEFI: LENOVO v: GZCN18WW date: 04/20/2021 
Battery:   ID-1: BAT0 charge: 57.5 Wh (100.0%) condition: 57.5/61.0 Wh (94.3%) volts: 17.3 min: 15.4 model: Sunwoda L19D4PH3 
           type: Li-poly serial: <filter> status: Full cycles: 23 
CPU:       Info: 8-Core model: AMD Ryzen 9 5900HX with Radeon Graphics bits: 64 type: MT MCP arch: Zen 3 family: 19 (25) 
           model-id: 50 (80) stepping: N/A microcode: A50000B cache: L2: 4 MiB 
           flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm bogomips: 105440 
           Speed: 2091 MHz min/max: 1200/3300 MHz boost: enabled Core speeds (MHz): 1: 2091 2: 1397 3: 1198 4: 1197 5: 1197 
           6: 1197 7: 1197 8: 1197 9: 1197 10: 1197 11: 1197 12: 1196 13: 2652 14: 1197 15: 1196 16: 1197 
           Vulnerabilities: Type: itlb_multihit status: Not affected 
           Type: l1tf status: Not affected 
           Type: mds status: Not affected 
           Type: meltdown status: Not affected 
           Type: spec_store_bypass mitigation: Speculative Store Bypass disabled via prctl and seccomp 
           Type: spectre_v1 mitigation: usercopy/swapgs barriers and __user pointer sanitization 
           Type: spectre_v2 mitigation: Full AMD retpoline, IBPB: conditional, IBRS_FW, STIBP: always-on, RSB filling 
           Type: srbds status: Not affected 
           Type: tsx_async_abort status: Not affected 
Graphics:  Device-1: Advanced Micro Devices [AMD/ATI] vendor: Lenovo driver: amdgpu v: kernel bus-ID: 03:00.0 
           chip-ID: 1002:1638 class-ID: 0300 
           Device-2: IMC Networks Integrated Camera type: USB driver: uvcvideo bus-ID: 1-3:2 chip-ID: 13d3:5419 class-ID: fe01 
           serial: <filter> 
           Display: x11 server: X.Org 1.20.10 driver: loaded: amdgpu,ati unloaded: modesetting alternate: fbdev,vesa 
           display-ID: :0.0 screens: 1 
           Screen-1: 0 s-res: 2880x1800 s-dpi: 96 s-size: 762x476mm (30.0x18.7") s-diag: 898mm (35.4") 
           Monitor-1: eDP res: 2880x1800 hz: 90 dpi: 242 size: 302x188mm (11.9x7.4") diag: 356mm (14") 
           OpenGL: renderer: AMD RENOIR (DRM 3.40.0 5.10.23-1-MANJARO LLVM 11.1.0) v: 4.6 Mesa 20.3.4 direct render: Yes 
Audio:     Device-1: Advanced Micro Devices [AMD/ATI] vendor: Lenovo driver: snd_hda_intel v: kernel bus-ID: 03:00.1 
           chip-ID: 1002:1637 class-ID: 0403 
           Device-2: Advanced Micro Devices [AMD] Raven/Raven2/FireFlight/Renoir Audio Processor vendor: Lenovo driver: N/A 
           alternate: snd_pci_acp3x, snd_rn_pci_acp3x bus-ID: 03:00.5 chip-ID: 1022:15e2 class-ID: 0480 
           Device-3: Advanced Micro Devices [AMD] Family 17h HD Audio vendor: Lenovo driver: snd_hda_intel v: kernel 
           bus-ID: 03:00.6 chip-ID: 1022:15e3 class-ID: 0403 
           Sound Server-1: ALSA v: k5.10.23-1-MANJARO running: yes 
           Sound Server-2: JACK v: 0.125.0 running: no 
           Sound Server-3: PulseAudio v: 14.2 running: yes 
           Sound Server-4: PipeWire v: 0.3.23 running: no 
Network:   Device-1: Realtek vendor: Lenovo driver: N/A port: 2000 bus-ID: 01:00.0 chip-ID: 10ec:8852 class-ID: 0280 
           IF-ID-1: enp3s0f4u1 state: unknown speed: N/A duplex: N/A mac: <filter> 
Bluetooth: Device-1: Xiaomi Mi/Redmi series (RNDIS) type: USB driver: rndis_host v: kernel bus-ID: 3-1:6 chip-ID: 2717:ff80 
           class-ID: 0a00 serial: <filter> 
           Report: This feature requires one of these tools: hciconfig/bt-adapter 
           Device-2: Realtek Bluetooth Radio type: USB driver: btusb v: 0.8 bus-ID: 3-4:2 chip-ID: 0bda:4852 class-ID: e001 
           serial: <filter> 
Drives:    Local Storage: total: 1011.15 GiB used: 0 KiB (0.0%) 
           SMART Message: Unable to run smartctl. Root privileges required. 
           ID-1: /dev/nvme0n1 maj-min: 259:0 vendor: Western Digital model: PC SN730 SDBPNTY-1T00-1101 size: 953.87 GiB 
           block-size: physical: 512 B logical: 512 B speed: 31.6 Gb/s lanes: 4 rotation: SSD serial: <filter> rev: 11190001 
           temp: 35.9 C scheme: GPT 
           ID-2: /dev/sda maj-min: 8:0 type: USB vendor: SanDisk model: Ultra USB 3.0 size: 57.28 GiB block-size: 
           physical: 512 B logical: 512 B serial: <filter> rev: 1.00 scheme: MBR 
           SMART Message: Unknown USB bridge. Flash drive/Unsupported enclosure? 
Partition: Message: No Partition data was found. 
Swap:      Alert: No Swap data was found. 
Sensors:   Message: No sensors data was found. Is sensors configured? 
Info:      Processes: 315 Uptime: 13m wakeups: 1 Memory: 13.59 GiB used: 2.36 GiB (17.4%) Init: systemd v: 247 tool: systemctl 
           Compilers: gcc: N/A Packages: pacman: 1135 lib: 299 flatpak: 0 Shell: Bash v: 5.1.0 running-in: xfce4-terminal 
           inxi: 3.3.03

sudo parted -l


[manjaro@manjaro ~]$ sudo parted -l
Model: SanDisk Ultra USB 3.0 (scsi)
Disk /dev/sda: 61.5GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End     Size    Type     File system  Flags
 2      2521MB  2525MB  4194kB  primary               esp


Model: WDC PC SN730 SDBPNTY-1T00-1101 (nvme)
Disk /dev/nvme0n1: 1024GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system  Name  Flags
 1      2097kB  317MB   315MB   fat32              boot, esp
 2      317MB   1024GB  1024GB               root

sudo efibootmgr -v

[manjaro@manjaro ~]$ sudo efibootmgr -v
BootCurrent: 0016
Timeout: 0 seconds
BootOrder: 0018,0016,0017,0000,0001,0012,0013,0014,0015
Boot0000* manjaro	HD(1,GPT,7fe01518-9921-cf44-94d9-1aca43959821,0x1000,0x96000)/File(\EFI\manjaro\grubx64.efi)
Boot0001* Windows Boot Manager	HD(1,GPT,24f4060f-3ad1-49be-8caf-67ac5a451f5a,0x800,0x82000)/File(\EFI\Microsoft\Boot\bootmgfw.efi)WINDOWS.........x...B.C.D.O.B.J.E.C.T.=.{.9.d.e.a.8.6.2.c.-.5.c.d.d.-.4.e.7.0.-.a.c.c.1.-.f.3.2.b.3.4.4.d.4.7.9.5.}....................
Boot0010  Setup	FvFile(721c8b66-426c-4e86-8e99-3457c46ab0b9)
Boot0011  Boot Menu	FvFile(86488440-41bb-42c7-93ac-450fbf7766bf)
Boot0012* NVMe: WDC PC SN730 SDBPNTY-1T00-1101         	PciRoot(0x0)/Pci(0x2,0x4)/Pci(0x0,0x0)/NVMe(0x1,00-1B-44-4A-46-23-C9-61)....2.LN........
Boot0013* ATA HDD:	VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,91af625956449f41a7b91f4f892ab0f600)
Boot0014* ATA HDD1:	VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,91af625956449f41a7b91f4f892ab0f601)
Boot0015* ATAPI CD:	VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,aea2090adfde214e8b3a5e471856a354)
Boot0016* USB HDD: SanDisk Ultra USB 3.0	PciRoot(0x0)/Pci(0x8,0x1)/Pci(0x0,0x3)/USB(5,0)3.!..3.G..A.....
Boot0017* USB FDD:	VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,6ff015a28830b543a8b8641009461e49)
Boot0018* USB CD:	VenMsg(bc7838d2-0f82-4d60-8316-c068ee79d25b,86701296aa5a7848b66cd49dd3ba6a55)

and

[manjaro@manjaro ~]$ 
test -d /sys/firmware/efi && echo efi || echo bios
efi

and lsblk

manjaro@manjaro ~]$ lsblk -f
NAME        FSTYPE      FSVER            LABEL            UUID                                 FSAVAIL FSUSE% MOUNTPOINT
loop0       squashfs    4.0                                                                          0   100% /run/miso/sfs/livefs
loop1       squashfs    4.0                                                                          0   100% /run/miso/sfs/mhwdfs
loop2       squashfs    4.0                                                                          0   100% /run/miso/sfs/desktopfs
loop3       squashfs    4.0                                                                          0   100% /run/miso/sfs/rootfs
sda         iso9660     Joliet Extension MANJARO_XFCE_210 2021-03-18-19-01-36-00                     0   100% /run/miso/bootmnt
├─sda1      iso9660     Joliet Extension MANJARO_XFCE_210 2021-03-18-19-01-36-00                              
└─sda2      vfat        FAT12            MISO_EFI         EF87-197E                                           
nvme0n1                                                                                                       
├─nvme0n1p1 vfat        FAT32            NO_LABEL         2802-502B                                           
└─nvme0n1p2 crypto_LUKS 1
2

when booting without USB stick, I decrypt the HDD, but then I land directly in the efi/bios menu.

3

Hi @berend,

I’d recommend booting from the USB stick, entering a chroot environment, and running the update again.

How to enter a chroot environment.

  1. Ensure you’ve got a relatively new ISO or at least one with an LTS kernel.

  2. Write/copy/dd the ISO to a USB thumb drive.

  3. When done, boot with the above mentioned USB thumb drive into the live environment.

  4. Once booted, open a terminal and enter the following command to enter the chroot encironment:

manjaro-chroot -a
  1. If you have more than one Linux installation, select the correct one to use from the list provided.

When done, you should now be in the chroot environment.

But, be careful, as you’re now in an actual root environment on your computer, so any changes you make will persist after a restart.

  1. From the chroot environment, run the update again, and hope nothing goes wrong:
pamac upgrade --enable-downgrade --aur --devel
  1. If all goes well, and there aren’t any errors, exit the chroot environment:
exit

  1. Followed by rebooting, doing a song, a dance, holding your thumbs and hoping for the best.

  2. If all went well, everything should be good again.

Hope this helps!

Edit:

Just saw you mention the drive is encrypted. Well, in theory, it should still work. However, I’ve no experience with it, so it might now, in which case…well, in which case, I don’t know. Sorry.

1 Like
4

You need to finish/redo the update (in chroot). And that’s all, probably.

5

I did use

manjaro-chroot

but not with a recent iso, because there was maintenance earlier. Will follow the steps above again with a newer iso

6

Doesn’t matter where you chroot from.

7

Not true. a Supported kernel is necessary. And considering this update drops 5.17 it’s important to use something that’s supported. Else I suspect you might have trouble.

8

in the not booting Manjaro installation, I was on Kernel 5.18, the best iso I could find has 5.15.

Hopefully good enough…

9

5.15 being LTS, I’d say it is, yes.

10

Supported by what? You can throw in debian live iso and chroot from there if you wish.

11

Something that’ll require the kernel version to work.

Good luck trying to install the nvidia drivers for a non-supported kernel version, for example.

12

You mean like this?

[debian /]# uname -a
Linux debian 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux
[debian /]# mhwd-kernel -li
Currently running: 5.10.0-9-amd64 (linux510)
The following kernels are installed in your system:
   * linux518
   * linux519
[debian /]# pacman -S linux519-nvidia-470xx
resolving dependencies...
looking for conflicting packages...

[...]

:: Running post-transaction hooks...
(1/7) Creating system user accounts...
Creating group 'nvidia-persistenced' with GID 143.
Creating user 'nvidia-persistenced' (NVIDIA Persistence Daemon) with UID 143 and GID 143.
(2/7) Reloading system manager configuration...
  Skipped: Running in chroot.
(3/7) Reloading device manager configuration...
  Skipped: Running in chroot.
(4/7) Arming ConditionNeedsUpdate...
(5/7) Updating module dependencies...
(6/7) Updating Kernel initcpios for Nvidia-DRM...

[...]

==> Image generation successful
(7/7) Updating the desktop file MIME type cache...
13

with the 5.15 iso, I did this (exact c&p):

boot into usb stick, open shell

su
cryptsetup open --type luks /dev/nvme0n1p2 crypto_LUKS
mount /dev/mapper/crypto_LUKS /mnt
mount /dev/nvme0n1p1 /mnt/boot/efi
manjaro-chroot /mnt

Note:

manjaro-chroot -a

does not work. Fails with no linux partitions found

This drops me into a sh shell, there:

# sudo grub-install --target="x86_64-efi" --efi-directory="/boot/efi" --bootloader-id=manjaro --recheck
Installing for x86_64-efi platform.
Installation finished. No error reported.

# sudo update-grub
Generating grub configuration file ...
Found theme: /usr/share/grub/themes/manjaro/theme.txt
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
ERROR: mkdir /var/lock/dmraid
grub-probe: error: cannot find a GRUB drive for /dev/sda1.  Check your device.map.
grub-probe: error: cannot find a GRUB drive for /dev/sda1.  Check your device.map.
Adding boot menu entry for UEFI Firmware Settings ...
Found memtest86+ image: /boot/memtest86+/memtest.bin
done

I run the same command again, because of the lock file. This happens everytime, btw.

sudo update-grub
Generating grub configuration file ...
Found theme: /usr/share/grub/themes/manjaro/theme.txt
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
grub-probe: error: cannot find a GRUB drive for /dev/sda1.  Check your device.map.
grub-probe: error: cannot find a GRUB drive for /dev/sda1.  Check your device.map.
Adding boot menu entry for UEFI Firmware Settings ...
Found memtest86+ image: /boot/memtest86+/memtest.bin
done

# sync

# exit

back in the previous shell, there:

sudo reboot -h now

…and again, I am landing in the efi menu after opening the encrypted HDD. Not booting into manjaro :sleepy:

14

Why aren’t you running pacman -Syu again? You don’t need to do anything else.

15

Why aren’t you running pacman -Syu again? You don’t need to do anything else.

Was on Ubuntu/Mint so far, this is my first distro with pacman. I did/do not know, that this would have helped.

At what point do I run this? Just boot from usb, open shell and then?

16

No, in chroot, same as you did other commands.

Well, you said yourself that system crashed during the update, so it’s safe to assume that update didn’t finish properly and you are probably left without any kernels.

17

not much happens:

sh-5.1# pacman -Syu
:: Synchronizing package databases...
error: failed to synchronize all databases (unable to lock database)
sh-5.1# rm /var/lib/pacman/db.lck
sh-5.1# pacman -Syu
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
 multilib is up to date
:: Starting full system upgrade...
 there is nothing to do
sh-5.1#
18

What’s output of ls /boot?

19 
sh-5.1# ls -lha /boot
total 80K
drwxr-xr-x  5 root root 4.0K Aug  8 11:44 .
drwxr-xr-x 17 root root 4.0K Jun 13 15:15 ..
-rw-r--r--  1 root root  50K Jul 15 16:22 amd-ucode.img
drwxr-xr-x  3 root root 4.0K Jan  1  1970 efi
drwxr-xr-x  6 root root 4.0K Aug  8 18:51 grub
-rw-r--r--  1 root root   22 Aug  3 13:18 linux515-x86_64.kver
-rw-r--r--  1 root root   22 Aug  3 13:17 linux518-x86_64.kver
drwxr-xr-x  2 root root 4.0K Dec 20  2021 memtest86+
20

Yeah, kernels and initramfs images missing.
Try installing those two kernels again (add headers too if you need them):

pacman -S linux515 linux518