Cannot boot after updates

Hi, first time user here.
I have Pinebook Pro from before Manjaro was the default. I downloaded Manjaro 20.08 KDE, booted from SD card, did dd to eMMC. I was able to boot from eMMC, all looked fine. Then I did system update using GUI and after reboot I get green LED but black screen no matter how long I wait. I cannot boot from SD card in such state at all.
So I went through disabling eMMC, booting from SDcard again, dd to emmc again. I now have a working Manjaro I tried to boot from eMMC couple of times, all fine. So my question is, how do I make sure after I do an update again I would be able to boot and not have to disassemble PBP again to be able to boot from SD?
I have zero experience with Manjaro, I am used to Ubuntu and Fedora.
Thank you

Thats really hard to help you with, when we don’t know what went wrong.

The most obvious one is that your /boot/extlinux/extlinux.conf file is getting changed to a non-working state for some reason.

Check if you have a extlinux.conf.pacnew|pacsave file after you update, and then make sure the regular one is correct before you reboot.

I was hoping there was some known problem… 161 updates to apply, will see how it goes this time and will check extlinux.conf. However, it disabled SDcard boot as well last time…

extlinux.conf was not changed by updates at all. And it is not bootable, neither from eMMC nor from SD card

What is the content of the extlinux.conf file?

LABEL Manjaro ARM
KERNEL /Image
FDT /dtbs/rockchip/rk3399-pinebook-pro.dtb
APPEND initrd=/initramfs-linux.img console=tty1 console=ttyS2,1500000 root=LABEL=ROOT_MNJRO rw rootwait bootsplash.bootfile=bootsplash-themes/manjaro/bootsplash

Yeah, that looks fine. Is your /boot getting mounted?

How could I tell? I have just a black screen

And you don’t have any SD or USB sticks in it?

No, nothing is connected

I have no idea then. Sorry. :frowning:

:frowning: As SD card boot does not work, wouldn’t that indicate update overwrote u-boot or something like that? Do I even make sense?

It would indicate that, but pacman does not overwrite uboot on package update.

Writing a new uboot requires the user to this manually, exactly for this reason. :slight_smile:

OK, I spent many hours and did many re-installs today.

  1. I downloaded 20.08 image but many packages were 20.04 (dolphin comes to mind), is that expected?
  2. I upgraded uboot package with pacman where I saw the instructions how to dd new uboot, I did that and reboot went fine
  3. I started to upgrade packages manually, just to see what breaks it and I cannot reboot after I upgraded linux and headers packages. And again, also SD card boot does not work. Whether trying to boot from eMMC or SD I get just black screen and green LED.

Any idea how to recover? I can boot SD card when I disable eMMC switch, then enable it, do the commands to make eMMC available etc, I can access data there.

Yes. They are different set of packages, following a different versioning.

That’s good. So it’s not a uboot issue.

What’s the versions of the packages you update to, that does not work for you?
(5.8.6 is the current ones in stable branch)

What’s the versions of the packages you update to, that does not work for you?
(5.8.6 is the current ones in stable branch)

How can I check it when I mount filesystem from SD boot? Or even better would be some pacman history file.

You can probably do ls -al $MOUNTPOINT/var/cache/pacman/pkg/linux* to see the last version it has downloaded.

You can also check the pacman log file at $MOUNTPOINT/var/log/pacman.log.

5.8.6-1

[2020-09-15T20:23:29+0200] [PACMAN] Running 'pacman -S libutil-linux linux util-linux'
[2020-09-15T20:23:32+0200] [ALPM] transaction started
[2020-09-15T20:23:32+0200] [ALPM] upgraded libutil-linux (2.36-1 -> 2.36-3)
[2020-09-15T20:23:32+0200] [ALPM] upgraded util-linux (2.36-1 -> 2.36-3)
[2020-09-15T20:23:41+0200] [ALPM] upgraded linux (5.8.1-1 -> 5.8.6-1)
[2020-09-15T20:23:41+0200] [ALPM] transaction completed
[2020-09-15T20:23:41+0200] [ALPM] running '20-systemd-sysusers.hook'...
[2020-09-15T20:23:41+0200] [ALPM] running '30-systemd-daemon-reload.hook'...
[2020-09-15T20:23:42+0200] [ALPM] running '30-systemd-udev-reload.hook'...
[2020-09-15T20:23:42+0200] [ALPM] running '30-systemd-update.hook'...
[2020-09-15T20:23:42+0200] [ALPM] running '60-depmod.hook'...
[2020-09-15T20:23:42+0200] [ALPM] running '60-linux.hook'...
[2020-09-15T20:23:48+0200] [ALPM] running '90-linux.hook'...
[2020-09-15T20:23:48+0200] [ALPM-SCRIPTLET] ==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
[2020-09-15T20:23:48+0200] [ALPM-SCRIPTLET]   -> -k 5.8.6-1-MANJARO-ARM -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
[2020-09-15T20:23:48+0200] [ALPM-SCRIPTLET] ==> Starting build: 5.8.6-1-MANJARO-ARM
[2020-09-15T20:23:49+0200] [ALPM-SCRIPTLET]   -> Running build hook: [base]
[2020-09-15T20:23:50+0200] [ALPM-SCRIPTLET]   -> Running build hook: [udev]
[2020-09-15T20:23:51+0200] [ALPM-SCRIPTLET]   -> Running build hook: [autodetect]
[2020-09-15T20:23:51+0200] [ALPM-SCRIPTLET]   -> Running build hook: [modconf]
[2020-09-15T20:23:51+0200] [ALPM-SCRIPTLET]   -> Running build hook: [block]
[2020-09-15T20:23:52+0200] [ALPM-SCRIPTLET]   -> Running build hook: [filesystems]
[2020-09-15T20:23:52+0200] [ALPM-SCRIPTLET]   -> Running build hook: [keyboard]
[2020-09-15T20:23:53+0200] [ALPM-SCRIPTLET]   -> Running build hook: [fsck]
[2020-09-15T20:23:53+0200] [ALPM-SCRIPTLET]   -> Running build hook: [bootsplash-manjaro]
[2020-09-15T20:23:53+0200] [ALPM-SCRIPTLET] ==> Generating module dependencies
[2020-09-15T20:23:53+0200] [ALPM-SCRIPTLET] ==> Creating gzip-compressed initcpio image: /boot/initramfs-linux.img
[2020-09-15T20:23:56+0200] [ALPM-SCRIPTLET] ==> Image generation successful
[2020-09-15T20:24:26+0200] [PACMAN] Running 'pacman -S linux-api-headers'
[2020-09-15T20:24:29+0200] [ALPM] transaction started
[2020-09-15T20:24:30+0200] [ALPM] upgraded linux-api-headers (5.6.11-1 -> 5.8-1)
[2020-09-15T20:24:30+0200] [ALPM] transaction completed
[2020-09-15T20:24:30+0200] [ALPM] running '30-systemd-update.hook'...

I’d suggest you check what is mounted at /boot.

With manjaro installed to both the eMMC and SD card, there are duplicate boot partition labels. On my pinebook pro, when booting from SD, the eMMC boot partition gets mounted. This seems to work fine until the kernel is updated on the SD card install. Then the new kernel and modules get written to eMMC which can break both the eMMC and SD installs.

The fix is to make the fstab more specific by either changing the labels or using UUID.

1 Like

Hah, found it. Turned out, there was some older u-boot written in SPI flash.

I flashed new u-boot following this Releases · pcm720/u-boot-build-scripts · GitHub and now all is fine, I have fully upgraded Manjaro running from eMMC and I can boot from SD as well.

Thank you for your help.