Updating kernel resulted in missing distros on Grub menu

@Mithradates provide

pacman -Qkk os-prober
ls /etc/grub.d

Did you move/resize partitions for Manjaro install?
And are you the same user as @Mithraidates ?

@bilingual @bebugz

When you update your kernel or install a new one, update-grub is always run after that automatically.

It could happen that when you install a new kernel you have to run update-grub again manually for your other OS-es to be picked up.

To be clear here…
If you run 4.9 and move to 4.12 you are not updating - you are installing a new kernel.
If you update 4.12.8-2 to 4.12.10-1 then that would be updating your kernel.
Also 4.11 is EOL [end of life]. https://www.kernel.org/

UPDATE: What an unpleasant situation! The maintainers even care about this? This is silly, installing a new kernel and suddenly losing the option of using the other OSes. At least show a message saying: "The new kernel has been installed. Now run ‘sudo update-grub’ from the console"
This happened to me also, but luckily I knew what to do. But what about those who don’t know? Do they have to ask for a solution on forums just to install a new kernel?

It could happen that when you install a new kernel […]

Yep! Quite a lot, as you can see. But thanks for your scrupulous info!

Still, @muser, will you consider an extended info, if you don’t mind, on this one: When you update your kernel or install a new one, update-grub is always run after that automatically. It could happen that when you install a new kernel you have to run update-grub again manually for your other OS-es to be picked up.
Why? This is BS! I say it because this struck me a few months ago when I started using Linux. I had to delete the entire disk (two Linux distros plus the one freshly installed) when the Grub menu showed only the last installed distro. I was lost there. Luckily those distros were new, so I didn’t have a problem deleting them. More luckily, after that situation I switched to Ubuntu, and Ubuntu never had this problem (although it was Ubuntu alongside WinXP), not even when it went from kernel v3 to kernel v4.

So, the real problem is what Mithradates and I had, not the semantic versioning of Linux kernel, being that an upgrade, an update, a downgrade or a ‘downdate’ or whatever you want to call it. In conclusion, when the kernel changes, no one should go through that misleading situation, especially those new to Linux.

It does matter whether you update your existing kernel or install a new one via the Manjaro Settings Manager (MSM). Update-grub is always run, that’s not the issue, but it seems that when installing a new kernel, MSM doesn’t run the os-prober part of it which would pick up the other distros.

Although it is an issue, I don’t think that those who are new to Linux should start right away with installing a new kernel, or even start with Manjaro for that matter, but if they do, they can always ask for help here which will kindly be provided.

When os-prober is not installed no other OS can be detected. update-grub is handled by a hook and should run after every kernel update. I’ve no issues with my dual setup having Manjaro 32bit and 64bit installed on one system for maintaining both architectures.

It also happened to me before, that even though I had os-prober, when installing a new kernel via MSM, it wiped out other disro entries, which I could fix with manually running update-grub again. It’s like MSM doesn’t handle the os-prober part properly when updating grub.

1 Like

As @philm mention

We have occasions where os-prober is not installed - thus blocking the detection of other available systems.

That said - I have had rare occasions where I had to explicit install os-prober to accomodate for other os’s installed.

Some users - I am not suggesting OP is one - has a habit of removing everything they think is not useful. And if doing so they might experience issues like OP. But that is by no means reasonable to accuse Manjaro for being naughty not detecting other present os’s.

I would like to assume you know your *nix but wouldn’t it be appropriate to point to the configuration file like

sudo grub-mkconfig -o /boot/grub/grub.cfg

I’ve been using Manjaro almost 2 years. Installing a new kernel used to work, that is it would update-grub and find the other distros. Finding the other distros stopped working maybe 6 months ago. This problem has been mentioned occasionally since then, but remains unfixed and apparently “unverified”. It’s an odd problem, because running update-grub after installing a new kernel does find the other distros. For that to work, os-prober has to be installed and enabled.

1 Like

Last year we switched from calling update-grub with the install files of each kernel to use a alpm-hook. See also in our git history. On our end we see no issues.

I reckon the issue - have heard it on grape-wine :slight_smile:

One thing I think might be common is the presence of /etc/lsb-release.

From what I have heard/read is that in the case of a missing /etc/lsb_release file the detection of other os might fail - but please bear with me - I am just throwing my thoughts without investigating further :slight_smile:

Simply execute os-prober and see what he finds:

sudo os-prober
/dev/sda5:Manjaro Linux (17.0.3):ManjaroLinux:linux

Based on that our bootloader adds the OS.

1 Like

That looks to be equivalent and a little cleaner. Grub-update clearly runs, the new manjaro kernel appears in the grub menu. Why wouldn’t os-prober run from the hook?

Here is my results

$ time sudo os-prober
/dev/sda1@/EFI/Microsoft/Boot/bootmgfw.efi:Windows Boot Manager:Windows:efi
/dev/sda10:Linux Mint 18.2 Sonya (18.2):LinuxMint:linux
/dev/sda17:Manjaro Linux (17.0.1):ManjaroLinux:linux
/dev/sda20:Debian GNU/Linux 9 (stretch):Debian:linux

real    0m27.701s
user    0m10.294s
sys    0m5.292s

The vineyard is here. :grin:

But it is not that os-prober does not detect other OS’s, rather that it takes a long time to do so.

@jbMacAZ: Is this the os-prober output of Manjaro or from a different distro? When Manjaro is installed, our OS should manage all other OSs installed on your system, since we added the intel-ucode patch to os-prober and grub packages. When you do a normal kernel update our package managers update the packages first, then run a hook to update the initramfs images followed by regeneration of the grub boot menu. Here an example output:

phil@manjaro ~ $ sudo pacman -S linux414
warning: linux414-4.14rc0.e7d0c41-1 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (1) linux414-4.14rc0.e7d0c41-1

Total Installed Size:  98.30 MiB
Net Upgrade Size:       0.00 MiB

:: Proceed with installation? [Y/n] 
(1/1) checking keys in keyring                     [######################] 100%
(1/1) checking package integrity                   [######################] 100%
(1/1) loading package files                        [######################] 100%
(1/1) checking for file conflicts                  [######################] 100%
(1/1) checking available disk space                [######################] 100%
:: Running pre-transaction hooks...
(1/1) Remove DKMS modules
==> dkms remove vboxguest/5.1.26_OSE -k 4.14.0-1-MANJARO
Error! There is no instance of vboxguest 5.1.26_OSE
for kernel 4.14.0-1-MANJARO (x86_64) located in the DKMS tree.
==> dkms remove vboxhost/5.1.26_OSE -k 4.14.0-1-MANJARO
==> dkms remove broadcom-wl/6.30.223.271 -k 4.14.0-1-MANJARO
==> dkms remove rtl8188eu/r461.00b5f0d -k 4.14.0-1-MANJARO
:: Processing package changes...
(1/1) reinstalling linux414                        [######################] 100%
>>> Updating module dependencies. Please wait ...
:: Running post-transaction hooks...
(1/4) Install DKMS modules
==> dkms install vboxguest/5.1.26_OSE -k 4.14.0-1-MANJARO
Error! Bad return status for module build on kernel: 4.14.0-1-MANJARO (x86_64)
Consult /var/lib/dkms/vboxguest/5.1.26_OSE/build/make.log for more information.
==> dkms install vboxhost/5.1.26_OSE -k 4.14.0-1-MANJARO
==> dkms install broadcom-wl/6.30.223.271 -k 4.14.0-1-MANJARO
==> dkms install rtl8188eu/r461.00b5f0d -k 4.14.0-1-MANJARO
(2/4) Updating linux414 initcpios
==> Building image from preset: /etc/mkinitcpio.d/linux414.preset: 'default'
  -> -k /boot/vmlinuz-4.14-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-4.14-x86_64.img
==> Starting build: 4.14.0-1-MANJARO
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
  -> Running build hook: [plymouth]
  -> Running build hook: [resume]
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-4.14-x86_64.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux414.preset: 'fallback'
  -> -k /boot/vmlinuz-4.14-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-4.14-x86_64-fallback.img -S autodetect
==> Starting build: 4.14.0-1-MANJARO
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
  -> Running build hook: [plymouth]
  -> Running build hook: [resume]
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-4.14-x86_64-fallback.img
==> Image generation successful
(3/4) Updating Grub-Bootmenu
Generating grub configuration file ...
Found background: /usr/share/grub/background.png
Found Intel Microcode image
Found linux image: /boot/vmlinuz-4.14-x86_64
Found initrd image: /boot/initramfs-4.14-x86_64.img
Found initrd fallback image: /boot/initramfs-4.14-x86_64-fallback.img
Found linux image: /boot/vmlinuz-4.13-x86_64
Found initrd image: /boot/initramfs-4.13-x86_64.img
Found initrd fallback image: /boot/initramfs-4.13-x86_64-fallback.img
Found linux image: /boot/vmlinuz-4.12-x86_64
Found initrd image: /boot/initramfs-4.12-x86_64.img
Found initrd fallback image: /boot/initramfs-4.12-x86_64-fallback.img
Found linux image: /boot/vmlinuz-4.11-x86_64
Found initrd image: /boot/initramfs-4.11-x86_64.img
Found initrd fallback image: /boot/initramfs-4.11-x86_64-fallback.img
Found linux image: /boot/vmlinuz-4.10-x86_64
Found initrd image: /boot/initramfs-4.10-x86_64.img
Found initrd fallback image: /boot/initramfs-4.10-x86_64-fallback.img
Found linux image: /boot/vmlinuz-4.9-x86_64
Found initrd image: /boot/initramfs-4.9-x86_64.img
Found initrd fallback image: /boot/initramfs-4.9-x86_64-fallback.img
Found linux image: /boot/vmlinuz-4.8-x86_64
Found initrd image: /boot/initramfs-4.8-x86_64.img
Found initrd fallback image: /boot/initramfs-4.8-x86_64-fallback.img
Found linux image: /boot/vmlinuz-4.4-x86_64
Found initrd image: /boot/initramfs-4.4-x86_64.img
Found initrd fallback image: /boot/initramfs-4.4-x86_64-fallback.img
Found linux image: /boot/vmlinuz-4.1-x86_64
Found initrd image: /boot/initramfs-4.1-x86_64.img
Found initrd fallback image: /boot/initramfs-4.1-x86_64-fallback.img
Found linux image: /boot/vmlinuz-3.18-x86_64
Found initrd image: /boot/initramfs-3.18-x86_64.img
Found initrd fallback image: /boot/initramfs-3.18-x86_64-fallback.img
Found linux image: /boot/vmlinuz-3.16-x86_64
Found initrd image: /boot/initramfs-3.16-x86_64.img
Found initrd fallback image: /boot/initramfs-3.16-x86_64-fallback.img
Found linux image: /boot/vmlinuz-3.12-x86_64
Found initrd image: /boot/initramfs-3.12-x86_64.img
Found initrd fallback image: /boot/initramfs-3.12-x86_64-fallback.img
Found linux image: /boot/vmlinuz-3.10-x86_64
Found initrd image: /boot/initramfs-3.10-x86_64.img
Found initrd fallback image: /boot/initramfs-3.10-x86_64-fallback.img
Found Manjaro Linux (17.0.3) on /dev/sda5
Found memtest86+ image: /boot/memtest86+/memtest.bin
done
(4/4) Arming ConditionNeedsUpdate...
phil@manjaro ~ $ 

os-prober actions are these:

Found Manjaro Linux (17.0.3) on /dev/sda5
Found memtest86+ image: /boot/memtest86+/memtest.bin

On your end that should be displayed

Found Windows 10 on /dev/sda1
Found Linux Mint on /dev/sda10
Found Manjaro Linux (17.0.1) on /dev/sda17
Found Debian 9 on /dev/sda20

If not, some is still missing on your end. What happens if calling sudo update-grub alone?

My os-prober output is from Manjaro. I have manjaro on 3 systems and all use Manjaro’s grub as the master (avoids kernel panics when booting Manjaro.)

  1. windows10 and Manjaro Cinnamon dual boot (no custom kernels). (kabylake laptop)
  2. windows10/Manjaro Cinnamon/Mint18.2/Debian9 (skylake desktop, slow HDD)
  3. Manjaro-KDE/MX-linux-16.1/Mint Cinnamon 18.2/ +32bit Mint installed on a USB. (Baytrail 2-in-1 laptop with 32bit UEFI)

Here is the output from a pacman re-install 4.12

[john@Linux-Beast ~]$ sudo pacman -S linux412
warning: linux412-4.12.8-2 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (1) linux412-4.12.8-2

Total Installed Size:  97.54 MiB
Net Upgrade Size:       0.00 MiB

:: Proceed with installation? [Y/n] y
(1/1) checking keys in keyring                     [######################] 100%
(1/1) checking package integrity                   [######################] 100%
(1/1) loading package files                        [######################] 100%
(1/1) checking for file conflicts                  [######################] 100%
(1/1) checking available disk space                [######################] 100%
:: Processing package changes...
(1/1) reinstalling linux412                        [######################] 100%
>>> Updating module dependencies. Please wait ...
:: Running post-transaction hooks...
(1/3) Updating linux412 initcpios
==> Building image from preset: /etc/mkinitcpio.d/linux412.preset: 'default'
  -> -k /boot/vmlinuz-4.12-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-4.12-x86_64.img
==> Starting build: 4.12.8-2-MANJARO
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
  -> Running build hook: [resume]
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-4.12-x86_64.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux412.preset: 'fallback'
  -> -k /boot/vmlinuz-4.12-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-4.12-x86_64-fallback.img -S autodetect
==> Starting build: 4.12.8-2-MANJARO
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [keyboard]
  -> Running build hook: [keymap]
  -> Running build hook: [resume]
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-4.12-x86_64-fallback.img
==> Image generation successful
(2/3) Updating Grub-Bootmenu
Generating grub configuration file ...
Found background: /usr/share/grub/background.png
Found Intel Microcode image
Found linux image: /boot/vmlinuz-4.12-x86_64
Found initrd image: /boot/initramfs-4.12-x86_64.img
Found initrd fallback image: /boot/initramfs-4.12-x86_64-fallback.img
Found linux image: /boot/vmlinuz-4.9-x86_64
Found initrd image: /boot/initramfs-4.9-x86_64.img
Found initrd fallback image: /boot/initramfs-4.9-x86_64-fallback.img
Found Windows Boot Manager on /dev/sda1@/EFI/Microsoft/Boot/bootmgfw.efi
Found Linux Mint 18.2 Sonya (18.2) on /dev/sda10
Found Manjaro Linux (17.0.1) on /dev/sda17
Found Debian GNU/Linux 9 (stretch) on /dev/sda20
Adding boot menu entry for EFI firmware configuration\n
Found memtest86+ image: /boot/memtest86+/memtest.bin
done
(3/3) Arming ConditionNeedsUpdate...

What is this? -> “(3/3) Arming ConditionNeedsUpdate…”

I rarely install a new kernel with MSM but that is when the other OS’s get dropped from the grub menu. But running update-grub fixes that quickly.

See this output from the detail of Manjaro Settings Manager for installing kernel 4.13-r8168

The following packages will be installed:
linux-rt-manjaro
linux-rt-manjaro-r8168

Starting
downloading linux413-r8168-8.044.02-0.5-x86_64.pkg.tar.xz...
error: failed retrieving file 'linux413-r8168-8.044.02-0.5-x86_64.pkg.tar.xz' from mirror.dacentec.com : The requested URL returned error: 404 
<snip several other 404's>
error: failed retrieving file 'linux413-r8168-8.044.02-0.5-x86_64.pkg.tar.xz' from kibo.remi.lu : The requested URL returned error: 404
checking keyring...
checking package integrity...
loading package files...
checking for file conflicts...
checking available disk space...
:: Processing package changes...
installing linux413...
:: Processing package changes...
installing linux413...
>>> Updating module dependencies. Please wait ...
Optional dependencies for linux413
    crda: to set the correct wireless channels of your country [installed]
installing linux413-r8168...
Optional dependencies for linux413
    crda: to set the correct wireless channels of your country [installed]
installing linux413-r8168...
>>> The module r8168 conflicts with r8169. You can blacklist it with:
>>>  `echo "blacklist r8169" > /etc/modprobe.d/r8169_blacklist.conf`
>>> The module r8168 conflicts with r8169. You can blacklist it with:
>>>  `echo "blacklist r8169" > /etc/modprobe.d/r8169_blacklist.conf`
:: Running post-transaction hooks...
(1/3) Updating linux413 initcpios
:: Running post-transaction hooks...
(1/3) Updating linux413 initcpios
==> Building image from preset: /etc/mkinitcpio.d/linux413.preset: 'default'
-> -k /boot/vmlinuz-4.13-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-4.13-x86_64.img
==> Starting build: 4.13.0-1-MANJARO
-> Running build hook: [base]
-> Running build hook: [udev]
-> Running build hook: [autodetect]
-> Running build hook: [modconf]
-> Running build hook: [block]
-> Running build hook: [keyboard]
-> Running build hook: [keymap]
-> Running build hook: [resume]
-> Running build hook: [filesystems]
-> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-4.13-x86_64.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux413.preset: 'fallback'
-> -k /boot/vmlinuz-4.13-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-4.13-x86_64-fallback.img -S autodetect
==> Starting build: 4.13.0-1-MANJARO
-> Running build hook: [base]
-> Running build hook: [udev]
-> Running build hook: [modconf]
-> Running build hook: [block]
-> Running build hook: [keyboard]
-> Running build hook: [keymap]
-> Running build hook: [resume]
-> Running build hook: [filesystems]
-> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-4.13-x86_64-fallback.img
==> Image generation successful
(2/3) Updating Grub-Bootmenu
Generating grub configuration file ...
Found background: /usr/share/grub/background.png
Found Intel Microcode image
Found linux image: /boot/vmlinuz-4.13-x86_64
Found initrd image: /boot/initramfs-4.13-x86_64.img
Found initrd fallback image: /boot/initramfs-4.13-x86_64-fallback.img
Found linux image: /boot/vmlinuz-4.12-x86_64
Found initrd image: /boot/initramfs-4.12-x86_64.img
Found initrd fallback image: /boot/initramfs-4.12-x86_64-fallback.img
Found linux image: /boot/vmlinuz-4.9-x86_64
Found initrd image: /boot/initramfs-4.9-x86_64.img
Found initrd fallback image: /boot/initramfs-4.9-x86_64-fallback.img
Adding boot menu entry for EFI firmware configuration\n
Found memtest86+ image: /boot/memtest86+/memtest.bin
done
(3/3) Arming ConditionNeedsUpdate...


Done ...


Done ...

After rebooting, only Manjaro kernels appears. Either os-prober didn’t find anything or it wasn’t launched.

Edit: Anyone understand the highlighting in my post? It wasn’t in the preview.

This is what I was trying to point out, that the issue doesn’t seem to be with os-prober itself, but when it’s run (or not run) by MSM after installing a new kernel.
It works fine when updating a kernel or running update-grub manually.

MSM doesn’t use os-prober at all. We have to check it. See also here.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Forum kindly sponsored by Bytemark