[Pinebook Pro] 5.13 no longer boots from NVME

Looking at my Manjaro XFCE system on my SD card as well as my backup of Manjaro KDE Fusion 5.12 that I copied to eMMC, it appears that I might need to rename “vmlinux-5.13.0-MANJRO-ARM” to “Image”, because U-Boot does not support selecting multiple kernels like grub so the kernel is simply named “Image.”

Or maybe I can keep the old Image and initramfs.linux.img files and just edit extlinux.conf to point to the new files. That seems safer, because if it doesn’t boot, I can restore extlinux.conf and get the system to boot (I think).

This is the safest way to test out new kernels without uninstalling the old one.

mkinitcpio would not let me generate an initramfs with a new filename so I’m using the original name (initramfs-linux.img ) and, even though mkinitcpio seems to to permit a new kernel name, I’m using the original name “Image” too. (I saved the old files as .bak files.)

After generating the initramfs and rebooting, it’s not booting my new 5.13 kernel:

uname -ar
Linux pinebook 5.12.11-1-MANJARO-ARM #1 SMP Wed Jun 16 10:48:53 UTC 2021 aarch64 GNU/Linux
[calinb@pinebook ~]$ 

Maybe it’s because my mmcblk2 and nvme0n1 partitions have the same UUID (but different labels, which I changed):

lsblk -f
NAME         FSTYPE FSVER LABEL           UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
mmcblk2                                                                                       
├─mmcblk2p1  vfat   FAT16 MMCBT_MNJRO     DC5B-1D91                                           
└─mmcblk2p2  ext4   1.0   EMMC_ROOT_MNJRO a24b5d20-0aef-4e1d-a516-c5f272500ed3                
mmcblk2boot0                                                                                  
mmcblk2boot1                                                                                  
zram0                                                                                         [SWAP]
nvme0n1                                                                                       
├─nvme0n1p1  vfat   FAT16 BOOT_MNJRO      DC5B-1D91                                94M    56% /boot
└─nvme0n1p2  ext4   1.0   ROOT_MNJRO      a24b5d20-0aef-4e1d-a516-c5f272500ed3  850.3G     3% /

The labels in extlinux.conf match my nvme:

$ cat /boot/extlinux/extlinux.conf  
LABEL Manjaro ARM
KERNEL /Image
FDT /dtbs/rockchip/rk3399-pinebook-pro.dtb
APPEND initrd=/initramfs-linux.img console=tty1 console=ttyS2,1500000 root=LABEL=ROOT_MNJRO rw rootwait video=eDP-1:1920x1080@60 video=HDMI-A-1:1920x1080@60 bootsplash.bootfile=bootsplash-theme
s/manjaro/bootsplash

What should I do about the duplicate UUIDs? I could just change the UUIDs on my emmc but, if U-Boot uses UUIDs, where is the configuration for them?

I used gparted to reset the UUIDs on both mmcblk2 partitions, re-ran mkinitcpio (which generated a new initramfs-linux.img) but, upon reboot, uname -r still reports 5.12 is running.

Since making my UUIDs unique, I did a “make clean” and rebuilt the kernel to the end of the process

make -j$(nproc)
..
..
sudo mkinitcpio -p linux

but it still boots to the old 5.12 kernel:

$ uname -r
5.12.11-1-MANJARO-ARM

:unamused:

Maybe I should use makepkg -s, as you describe here:

Where does makepkg expect to find config (and is it “config” or “.config”)? Should config exist in the same folder as PKGBUILD? Where does it expect to find the .preset file (“linux.preset”)?

What steps must be completed after makepkg -s to install the kernel?

Well, I tried makepkg -s and got this:

...
...
patching file tools/testing/selftests/kvm/dirty_log_test.c
patching file tools/testing/selftests/kvm/lib/kvm_util.c
patching file tools/testing/selftests/kvm/lib/x86_64/processor.c
patching file tools/testing/selftests/kvm/set_memory_region_test.c
patching file tools/testing/selftests/kvm/steal_time.c
patching file tools/testing/selftests/kvm/x86_64/set_boot_cpu_id.c
patching file tools/testing/selftests/lkdtm/run.sh
patching file tools/testing/selftests/lkdtm/tests.txt
patching file tools/testing/selftests/net/forwarding/pedit_dsfield.sh
patching file tools/testing/selftests/net/forwarding/pedit_l4port.sh
patching file tools/testing/selftests/net/forwarding/skbedit_priority.sh
patching file tools/testing/selftests/net/tls.c
patching file tools/testing/selftests/powerpc/pmu/ebb/no_handler_test.c
patching file tools/testing/selftests/resctrl/README
patching file tools/testing/selftests/resctrl/resctrl_tests.c
patching file tools/testing/selftests/sgx/load.c
patching file tools/testing/selftests/splice/short_splice_read.sh
patching file tools/testing/selftests/tc-testing/plugin-lib/scapyPlugin.py
patching file tools/testing/selftests/timers/rtcpie.c
patching file tools/testing/selftests/vm/protection_keys.c
patching file virt/kvm/coalesced_mmio.c
==> ERROR: A failure occurred in prepare().
    Aborting...

Okay. That’s a lot of questions.

Makepkg it self does not use the config file. But it is listed in the source of the PKGBUILD. So the file is called what it’s called in the PKGBUILD (which in our case is config).

Yes. All the files mentioned in the source=() array should be in the same folder as the PKGBUILD, unless the source is a URL.

In the same folder as the PKGBUILD, just like config.

To install a package that was built with makepkg you run:

sudo pacman -U <path to package file>

According to the output: ==> ERROR: A failure occurred in prepare(). there was an error in the prepare step. So likely a patch failed to apply. But we can’t know for sure, without the entire output (you only pasted a couple of lines, which so not contain the actual error).

Yes–many questions but your answers helped me greatly! I’ll debug the prepare error later. It was from my Manjaro XFCE system on SD card. I went back to my Manjaro Fusion system on NVMe SSD and makepkg worked!

However, I don’t see any new Image or vmlinux or other new file in /boot. There is a ~/linux/linux-5.13/vmlinux file with a new timestamp. Do I need to copy it to /boot/Image ?

Did you do install the package or just build it?

I did

updpkgsums
makepkg -s

then I did

sudo pacman -U ~/linux/linux-5.13.4-2-aarch64.pkg.tar.zst
sudo pacman -U ~/linux/linux-headers-5.13.4-2-aarch64.pkg.tar.zst

Here’s the output from the linux-5.13.4-2 install:

$ sudo pacman -U ~/linux/linux-5.13.4-2-aarch64.pkg.tar.zst  
loading packages...
resolving dependencies...
looking for conflicting packages...

Packages (1) linux-5.13.4-2

Total Installed Size:   79.84 MiB
Net Upgrade Size:      -35.09 MiB

:: Proceed with installation? [Y/n] 
(1/1) checking keys in keyring                                                                                      [######################################################################] 100%
(1/1) checking package integrity                                                                                    [######################################################################] 100%
(1/1) loading package files                                                                                         [######################################################################] 100%
(1/1) checking for file conflicts                                                                                   [######################################################################] 100%
(1/1) checking available disk space                                                                                 [######################################################################] 100%
:: Processing package changes...
(1/1) upgrading linux                                                                                               [######################################################################] 100%
warning: /etc/mkinitcpio.d/linux.preset installed as /etc/mkinitcpio.d/linux.preset.pacnew
:: Running post-transaction hooks...
(1/4) Arming ConditionNeedsUpdate...
(2/4) Updating module dependencies...
(3/4) Updating linux module dependencies...
(4/4) Updating linux initcpios...
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
  -> -k 5.13.0-MANJARO-ARM -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
==> Starting build: 5.13.0-MANJARO-ARM
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux.img
==> Image generation successful
[calinb@pinebook linux]$ sudo pacman -U /home/calinb/linux/linux-headers-5.13.4-2-aarch64.pkg.tar.zst 
loading packages...
resolving dependencies...
looking for conflicting packages...

Packages (1) linux-headers-5.13.4-2

Total Installed Size:  83.99 MiB

:: Proceed with installation? [Y/n] 
(1/1) checking keys in keyring                                                                                      [######################################################################] 100%
(1/1) checking package integrity                                                                                    [######################################################################] 100%
(1/1) loading package files                                                                                         [######################################################################] 100%
(1/1) checking for file conflicts                                                                                   [######################################################################] 100%
(1/1) checking available disk space                                                                                 [######################################################################] 100%
:: Processing package changes...
(1/1) installing linux-headers                                                                                      [######################################################################] 100%
:: Running post-transaction hooks...
(1/3) Arming ConditionNeedsUpdate...
(2/3) Updating module dependencies...
(3/3) Updating linux module dependencies...

Okay. Did you change any file locations in the PKGBUILD before building it?

If you didn’t, the file you have in /boot/Image is the new one, since it will overwrite it.

The timestamp is wrong on /boot/Image. Can’t be the new one.

I think I maybe need to update the .preset file line in /etc/mkinitcpio.d/linux.preset

cat /etc/mkinitcpio.d/linux.preset
# mkinitcpio preset file for the '%PKGBASE%' package

ALL_config="/etc/mkinitcpio.conf"
ALL_kver="5.13.0-MANJARO-ARM"

PRESETS=('default')

#default_config="/etc/mkinitcpio.conf"
default_image="/boot/initramfs-linux.img"
#default_options=""

You have a .pacnew there, so move that over in it’s stead and rebuild the initramfs.

Okay. I did " sudo mv linux.preset.pacnew linux.preset"

and then:

 sudo mkinitcpio -p linux
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
  -> -k 5.13.4-2-MANJARO-ARM -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
==> Starting build: 5.13.4-2-MANJARO-ARM
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux.img
==> Image generation successful
 

But no “Image” or “vmlinux” file in /boot still:

ls -tla
total 83444
-rwxr-xr-x  1 root root  7208560 Jul 26 15:27 initramfs-linux.img
drwxr-xr-x  9 root root     4096 Jul 26 14:52 dtbs
drwxr-xr-x  2 root root     4096 Jul 26 11:57 extlinux
-rwxr-xr-x  1 root root  7208095 Jul 26 11:56 initramfs-linux.img.bak
-rwxr-xr-x  1 root root 30974464 Jul 26 11:56 Image.bak
-rwxr-xr-x  1 root root 31468032 Jul 26 11:54 Image.old
drwxr-xr-x  4 root root     4096 Jul 26 11:15 .Trash-1000
-rwxr-xr-x  1 root root   157980 May 23 10:04 idbloader.img
-rwxr-xr-x  1 root root  4194304 May 23 10:04 trust.img
-rwxr-xr-x  1 root root  4194304 May 23 10:04 uboot.img
drwxr-xr-x 17 root root     4096 May  5 15:01 ..
drwxr-xr-x  5 root root    16384 Dec 31  1969 .

I tried reinstalling and got an Image file warning

“warning: could not get file information for boot/Image”

loading packages...
warning: linux-5.13.4-2 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (1) linux-5.13.4-2

Total Installed Size:  79.84 MiB
Net Upgrade Size:       0.00 MiB

:: Proceed with installation? [Y/n] 
(1/1) checking keys in keyring                                                                                      [######################################################################] 100%
(1/1) checking package integrity                                                                                    [######################################################################] 100%
(1/1) loading package files                                                                                         [######################################################################] 100%
(1/1) checking for file conflicts                                                                                   [######################################################################] 100%
(1/1) checking available disk space                                                                                 [######################################################################] 100%
warning: could not get file information for boot/Image
:: Processing package changes...
(1/1) reinstalling linux                                                                                            [######################################################################] 100%
:: Running post-transaction hooks...
(1/4) Arming ConditionNeedsUpdate...
(2/4) Updating module dependencies...
(3/4) Updating linux module dependencies...
(4/4) Updating linux initcpios...
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
  -> -k 5.13.4-2-MANJARO-ARM -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
==> Starting build: 5.13.4-2-MANJARO-ARM
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux.img
==> Image generation successful

Okay–it did create a /boot/Image file. It has an earlier timestamp than I expected but installing the package did create it.

The system would not boot, however, and I put the PCI_MSI code back in there (I’ll double check.)

For now, I don’t think I need assistance. I’ll reboot to my SD card and try to debug the prepare error on that system. I need to confirm that I can build and install a kernel on that SD card system and also include the PCI_MSI code and successfully boot. Then I will return to the MVNe system.

Thanks for all your help, Strit. I’ll keep plugging away at it, because I’d like to confirm the PCI_MSI problem and file a bug report to linux dot org.

I did cp -a ~/home/linux to my SD card XFCE system. Then

pacman - U sudo pacman -U linux-5.13.4-2-aarch64.pkg.tar.zst

and rebooted and confirmed my new kernel was running from my SD card:

$ uname -r
5.13.4-2-MANJARO-ARM

I doubled-checked and my PCI_MSI edits are in place and also the line
"CONFIG_PCI_MSI_ARCH_FALLBACKS=y"

is in ~/linux/config

so my MSI-regresson kernel’s failure to boot on my NVMe SSD seems to not be related to PCI_MSI changes in 5.12 >> 5.13.

I can archive the three .c source files and the config file and send them to you, if you’re interested in having a look at them but here are the code insertions:

$ grep PCI_MSI ~/linux/config
CONFIG_PCI_MSI=y
CONFIG_PCI_MSI_IRQ_DOMAIN=y
CONFIG_PCI_MSI_ARCH_FALLBACKS=y
$ diff linux-5.13/drivers/pci/msi.c linux-5.13/drivers/pci/msi.c.bak
67,79c67
< 	struct msi_controller *chip = dev->bus->msi;
< 	int err;
< 
< 	if (!chip || !chip->setup_irq)
< 		return -EINVAL;
< 
< 	err = chip->setup_irq(chip, dev, desc);
< 	if (err < 0)
< 		return err;
< 
< 	irq_set_chip_data(desc->irq, chip);
< 
< 	return 0;
---
> 	return -EINVAL;
84,89d71
< 	struct msi_controller *chip = irq_get_chip_data(irq);
< 
< 	if (!chip || !chip->teardown_irq)
< 		return;
< 
< 	chip->teardown_irq(chip, irq);
94d75
< 	struct msi_controller *chip = dev->bus->msi;
97,98c78
< 	if (chip && chip->setup_irqs)
< 		return chip->setup_irqs(chip, dev, nvec, type);
---
> 
$ diff linux-5.13/drivers/pci/probe.c linux-5.13/drivers/pci/probe.c.bak
898d897
< 	bus->msi = bridge->msi;
1059d1057
< 	child->msi = parent->msi;
$ diff linux-5.13/drivers/pci/probe.c linux-5.13/drivers/pci/probe.c.bak
898d897
< 	bus->msi = bridge->msi;
1059d1057
< 	child->msi = parent->msi;
[calinb@PBP linux]$ diff linux-5.13/include/linux/pci.h linux-5.13/include/linux/pci.h.bak
542d541
< 	struct msi_controller *msi;
624d622
< 	struct msi_controller *msi;	/* MSI controller */

Thanks for all your help, Strit. At least I can build a kernel package and install it now. :slight_smile:

I successfully installed my test kernel package to my Manjaro XFCE SD card system and it boots from the SD card. However, it still does not boot from my NVMe SSD Manjaro Fusion system. (I have installed it twice and restored the NVMe Manjaro system from backup twice after it failed to boot.) It fails to boot from NVMe in the same manner as the updated package from the Manjaro repository (black screen and no boot) so there is something else wrong in 5.13 that causes the NVMe boot to fail.