How to save your Manjaro installation when it breaks

backup
boot
chroot

#1

Some of you might have had it happen, a Linux installation on your computer that breaks after installing certain software, after installing updates or after editing a configuration file.

Sometimes you mess up so bad, that the entire system doesn’t boot past Grub or doesn’t even boot Grub itself if you messed around with its configuration files.

So what can you do when you really messed up big time?

What if you didn’t make that Clonezilla backup of your system that you should have made in case of emergencies?

Here is a solution that has saved my system on a number of occasions, when it was so broken, that Manjaro or even Grub didn’t load and all that you could see was a black screen at bootup.

  1. Start the Manjaro live installation media from a usb-stick (or from a CD/DVD for example).

  2. Now you need to find out the name of the disk or partition on which your actual Manjaro installation resides. You do this by opening a terminal and typing:

lsblk

This now lists all your disks and partitions in the terminal window in a tree structure. Now you can determine on which partition you have Manjaro installed.
The name of the partition on which Manjaro is installed can look like this for example: sda2

  1. Now you need to mount the entire tree of your file system of your installed version of Manjaro into the live environment. This is very powerful stuff when you think about it. You will be able to access the file system of your Manjaro installation in the live environment!
    To do this, you need to choose a mount point in the file system of the live environment first. Typically, people use /mnt as the location to do this. So to accomplish this step, execute the following command in a terminal:
sudo mount /dev/sda2 /mnt

Now you’ve mounted the filesystem of sda2 into the /mnt mount-point. Powerful stuff!

  1. So now you have access to the files of your Manjaro installation, even in a graphical file manager like Thunar (at /mnt). However that’s not always enough to save your Manjaro installation. Sure you can edit configuration files now or even delete files that caused issues, but what you really want is full control. You want to be able to execute commands as if your were inside your actual Manjaro installation.
    This means that you need to change root to the new mount-point that you just created. And typically this is done with the chroot command, however the chroot command often doesn’t allow you to run the commands that are needed to repair a broken Linux installation.
    What you need, is a more powerful version of chroot. It’s called mhwd-chroot.
    You need to download and install it in your Live media from the Manjaro repositories:
sudo pacman -S mhwd-chroot
  1. Once installed, you can now change root to the new mount point. You do that from the menu (chroot) or by typing the following command in a terminal:
mhwd-chroot

This will present you the available system partitions you can change root to.

  1. Now a new terminal window will open. In this terminal window you can execute commands as if you were inside your actual Manjaro installation.

You can now uninstall software that caused your system to break or you can roll back updates or you can uninstall drivers like video-drivers that made your screen go black.
It’s powerful stuff, because you now have both access to your files and to a terminal which can control your Manjaro installation. The possibilities to repair things, fix issues and backup files are now endless. This and much more. All due to these few simple commands.
In short, here are the steps and commands in a nutshell:

  1. Boot into the Manjaro live installation media
  2. Open a terminal and type: lsblk
  3. Mount your disk into the live environment: sudo mount /dev/sda2 /mnt
  4. Download and install mhwd-chroot
  5. Change root into the new mount-point: mhwd-chroot
  6. Save your Manjaro installation using the chrooted terminal

One more thing I can add to this is that people who use PCIe storage devices, they need to download and install mhwd-chroot-extended (from the AUR), so that they can mount file systems from NVMe storage devices.


Change graphics drivers and now Manjaro doesn't start
Graphic driver freezes booting
Corrupted filesystem soon after a fresh install
General approach to updating?
[Stable Update] 2016-12-07 - Systemd, Kernels, Firefox, Thunderbird, Mesa, Python
Manjaro doesnt work after switching to kernel 4.1
[Stable Update] 2017-04-02 - Mesa-Stack, Kernels, Plasma, Firefox
Can't boot Deepin after update
After customizing panel, Manjaro won't boot any more
Can't login - black screen & chroot guide
[Stable Update] 2017-04-02 - Mesa-Stack, Kernels, Plasma, Firefox
UEFI Install Issues
[Stable Update] 2017-05-12 - Kernels, KDE, KDE Apps, Gnome, Browsers
Issue: manjaro Stuck during boot
[SOLVED] Network Manager not starting after update; libicuuc.so.58 seems to be cause
[Solved]Blank dark screen
[SOLVED] Libreadline.so.6 error while trying to chroot
[SOLVED] Couldn't start after update (UUID error)
Cant Boot to Manjaro
[Unstable Update] 2019-01-13 - Kernels, Cinnamon, Systemd, Readline, KDE Frameworks, Python
Some questions/thoughts about last week's "problem" stable update [2016-10-9]
[Stable Update] 2016-10-09 - Kernels, Manjaro Tools, Deepin, NM, LibreOffice, Steam
Full System Backup
Password MishMash
Error after maintenance via pacui. Should I reinstall?
System asks for ctrl+d after grub selection at boot; why?
[TOC] Boot-GRUB problems
Stuck at booting up : Started tlp system startup/shutdown / Starting user Manager for UID 120
[Testing Update i686] 2018-09-25 to 10-07 - systemd, Firefox
Radeon blacklight issue
[SOLVED] Manjaro self-rebooted during/after update and cannot boot anymore
Laptop stuck at emergency shell
After installing proprietary video drivers, my second hdmi monitor no longer works
Login in send me back to the login screen
Hallo, new novice noob, newbie and part time fool here
KDE has broken, extra kernel?
Does updateing manjaro will break the system?!
URL's from my cheat sheet
Manjaro 17.0-rc3 Installation fails
Primus could not load gpu driver
#2

Nice guide!

Suggested additions:

  • you could add instructions how to it with regular chroot. You just need to mount some extra stuff. See here: https://wiki.manjaro.org/index.php/Restore_the_GRUB_Bootloader
  • I think regular mhwd-chroot is in the repos, no aur access needed.
  • Since restoring bootloader is one of the most common reasons to chroot, you could mention that you need to uefi boot your live system in order to restore uefi bootloader.

Serious Upgrade Problem
#3

I think that instead of mhwd-chroot it’s possible to use manjaro-chroot, it should be preinstalled with every Manjaro release.


Updating Linux 414
#4

Yeah, the command
manjaro-chroot -a
would likely do the job if you have a standard file system. https://github.com/manjaro/manjaro-tools#6-manjaro-chroot
But mhwd-chroot is a script which uses the command which you would us in manual chrooting, it should work reliably for ages, I would guess.

But the guide has a different error: You don’t need to mount the partition like in point 3. in order to chroot it with mhwd-chroot and you don’t need to tell mhwd-chroot the mount point like in point 5. I suppose the OP tested it that way and it somehow worked. But the dev clearly wrote how to use it here https://classicforum.manjaro.org/index.php?topic=21472.0
Also the Wiki tells to just run the command mhwd-chroot and everithing goes automatically: https://wiki.manjaro.org/index.php?title=Restore_the_GRUB_Bootloader#Use_mhwd-chroot

Once again a warning to everybody to have a critical attitude towards third party tutorials.


#5

Yeah, the command
manjaro-chroot -a
would likely do the job if you have a standard file system.

What do you mean with standard filesystem? You speak about partition filesytem or filesystem hierarchy?
All the system in a single partition?
Only traditional filesystems like ext4/XFS?
(Probably reading the code I can understand it, but if you already know the answer… :wink: )


#6

I mean traditional filesystems like ext4, maybe xfs, or newer filesystems, but without subvolumes and similar things one could imagine. I tested both mhwd-chroot and manjaro-chroot with systems installed in btrfs subvolumes and it didn’t work. The point is that there mustn’t be anything between the partition and the / (root) directory.


#7

Thank you so infinetly much for this guide, I have taken to printing it…I never waste a tree, unless the message on the paper is life important. As this is…
Thank you so much
Melissa


#8

Useful, thanks.


#9

FYI The Command mhwd-chroot must be run sudo mhwd-chroot otherwise it gives you a crazy pssword loop.


#10

Note for anyone using a Btrfs partition, the process is slightly different. Here’s how a chrooted into my Btrfs system.
/dev/sda2 —> 47.0GB Btrfs /
/dev/sda3 —> 114GB Ext 4 /home

sudo mount /dev/sda2 /mnt 
sudo mount /dev/sda3 /mnt/@/home
manjaro-chroot /mnt/@
sudo pacman -Syyu
exit
reboot

Manjaro-chroot -a WILL NOT FIND ANYTHING, or throw an error, and in my case, it attempted to chroot to my Ubuntu partition. You need to do this manually to fix your manjaro install.


#11

Btrfs:

mount -o subvol=@ /dev/sda2 /mnt
manjaro-chroot /mnt 

Is the way to mount a btrfs volume :smiley: