For those running kernel 5.4 (and sometimes others) getting no relief from systemd boot fails

Ktastrophe · 23 2024 22:40

Hi all,

I spent the better part of these past two weeks trying to recover my system (or more specifically, kernels 4.19 and 5.4) that were no longer booting into the DE. Without a more plausible explanation, it was looking like AUR broke fakeroot, which somehow broke them… Anyway…
In these forums, I found lots of advice, from editing the grub file to force legacy cgroup1 support, enabling different compression formats in mkinitcpio, to video driver reinstallation–all of which, it turns out, didn’t solve my problem.
I then found a thread on GitHub that did:

github.com/systemd/systemd

Update systemd 255.7-1 to 256.1-1 => attempt restart Linux => segfault and kernel panic

opened 07:53AM - 04 Jul 24 UTC

closed 05:04PM - 20 Aug 24 UTC

one-way-ticket

bug 🐛 pid1 needs-reporter-feedback ❓

### systemd version the issue has been seen with 256.1-1-x86_64 ### Used distr…ibution Manjaro 24.0.3 ### Linux kernel version used 5.4.278-1-MANJARO (LTS) ### CPU architectures issue was seen on x86_64 ### Component systemd ### Expected behaviour you didn't see Linux Manjaro 24.0.3. kernel 5.4.278 (LTS), systemd 255.7-1. Installed in Virtualbox, works successfully. ### Unexpected behaviour you saw After update systemd 255.7-1 to 256.1-1: 1) restart system failed - segfault in libsystemd-shared-256.1-1.so, kernel panic, system stopped. 2) after reset - system loadind failed. ![Segfault](https://github.com/systemd/systemd/assets/62329477/a6ae13e6-44fe-43f5-bafd-bcaeb8e95fb9) ### Steps to reproduce the problem Problem after full update: `# pacman -Syu :: Synchronizing package databases... core is up to date extra is up to date community is up to date multilib is up to date :: Starting full system upgrade... resolving dependencies... looking for conflicting packages... Packages (4) lib32-systemd-256.1-1 systemd-256.1-1 systemd-libs-256.1-1 systemd-sysvcompat-256.1-1 Total Download Size: 10.28 MiB Total Installed Size: 37.15 MiB Net Upgrade Size: 1.61 MiB :: Proceed with installation? [Y/n] y ` ### Additional program output to the terminal or log subsystem illustrating the issue ```sh [VM-Test4 test_user]# pacman -Syu :: Synchronizing package databases... core is up to date extra is up to date community is up to date multilib is up to date :: Starting full system upgrade... resolving dependencies... looking for conflicting packages... Packages (4) lib32-systemd-256.1-1 systemd-256.1-1 systemd-libs-256.1-1 systemd-sysvcompat-256.1-1 Total Download Size: 10.28 MiB Total Installed Size: 37.15 MiB Net Upgrade Size: 1.61 MiB :: Proceed with installation? [Y/n] y :: Retrieving packages... systemd-256.1-1-x86_64 8.5 MiB 1678 KiB/s 00:05 [#############################################] 100% systemd-libs-256.1-1-x86_64 1129.9 KiB 1385 KiB/s 00:01 [#############################################] 100% lib32-systemd-256.1-1-x86_64 731.6 KiB 1025 KiB/s 00:01 [#############################################] 100% systemd-sysvcompat-256.1-1-x86_64 6.1 KiB 73.4 KiB/s 00:00 [#############################################] 100% Total (4/4) 10.3 MiB 1484 KiB/s 00:07 [#############################################] 100% (4/4) checking keys in keyring [#############################################] 100% (4/4) checking package integrity [#############################################] 100% (4/4) loading package files [#############################################] 100% (4/4) checking for file conflicts [#############################################] 100% (4/4) checking available disk space [#############################################] 100% :: Processing package changes... (1/4) upgrading systemd-libs [#############################################] 100% (2/4) upgrading lib32-systemd [#############################################] 100% (3/4) upgrading systemd [#############################################] 100% :: This is a systemd feature update. You may want to have a look at NEWS for what changed, or if you observe unexpected behavior: /usr/share/doc/systemd/NEWS (4/4) upgrading systemd-sysvcompat [#############################################] 100% :: Running post-transaction hooks... ( 1/12) Creating system user accounts... ( 2/12) Updating journal message catalog... ( 3/12) Reloading system manager configuration... ( 4/12) Reloading user manager configuration... ( 5/12) Updating udev hardware database... ( 6/12) Applying kernel sysctl settings... ( 7/12) Creating temporary files... ( 8/12) Reloading device manager configuration... ( 9/12) Arming ConditionNeedsUpdate... (10/12) Updating the MIME type database... (11/12) Updating linux initcpios... ==> Building image from preset: /etc/mkinitcpio.d/linux54.preset: 'default' ==> Using configuration file: '/etc/mkinitcpio.conf' -> -k /boot/vmlinuz-5.4-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-5.4-x86_64.img ==> Starting build: '5.4.278-1-MANJARO' -> Running build hook: [base] -> Running build hook: [udev] -> Running build hook: [autodetect] -> Running build hook: [modconf] -> Running build hook: [kms] ==> WARNING: No module containing the symbol 'drm_privacy_screen_register' found in: 'drivers/platform' -> Running build hook: [block] -> Running build hook: [keyboard] -> Running build hook: [keymap] -> Running build hook: [filesystems] ==> Generating module dependencies ==> Creating gzip-compressed initcpio image: '/boot/initramfs-5.4-x86_64.img' -> Early uncompressed CPIO image generation successful ==> Initcpio image generation successful ==> Building image from preset: /etc/mkinitcpio.d/linux54.preset: 'fallback' ==> Using configuration file: '/etc/mkinitcpio.conf' -> -k /boot/vmlinuz-5.4-x86_64 -c /etc/mkinitcpio.conf -g /boot/initramfs-5.4-x86_64-fallback.img -S autodetect ==> Starting build: '5.4.278-1-MANJARO' -> Running build hook: [base] -> Running build hook: [udev] -> Running build hook: [modconf] -> Running build hook: [kms] ==> WARNING: Possibly missing firmware for module: 'ast' ==> WARNING: No module containing the symbol 'drm_privacy_screen_register' found in: 'drivers/platform' -> Running build hook: [block] ==> WARNING: Possibly missing firmware for module: 'qed' ==> WARNING: Possibly missing firmware for module: 'bfa' ==> WARNING: Possibly missing firmware for module: 'qla1280' ==> WARNING: Possibly missing firmware for module: 'qla2xxx' -> Running build hook: [keyboard] -> Running build hook: [keymap] -> Running build hook: [filesystems] ==> Generating module dependencies ==> Creating gzip-compressed initcpio image: '/boot/initramfs-5.4-x86_64-fallback.img' -> Early uncompressed CPIO image generation successful ==> Initcpio image generation successful (12/12) Reloading system bus configuration... Job for dbus-broker.service failed. See "systemctl status dbus-broker.service" and "journalctl -xeu dbus-broker.service" for details. error: command failed to execute correctly [VM-Test4 test_user]# systemctl status dbus-broker.service ● dbus-broker.service - D-Bus System Message Bus Loaded: loaded (/usr/lib/systemd/system/dbus-broker.service; disabled; preset: disabled) Active: active (running) since Thu 2024-07-04 12:40:34 +06; 31min ago Invocation: 88a7395d66fa447182aa9a9c3a6b9037 TriggeredBy: ● dbus.socket Docs: man:dbus-broker-launch(1) Main PID: 685 (dbus-broker-lau) Tasks: 2 (limit: 9524) Memory: 7.9M CPU: 380ms CGroup: /system.slice/dbus-broker.service ├─685 /usr/bin/dbus-broker-launch --scope system --audit └─686 dbus-broker --log 4 --controller 9 --machine-id 321fc68d579f4a9fa7c5027fe9e52fe8 --max-bytes 536870912 --max-fds 4096 --max-matches 131072> Jul 04 13:09:52 VM-Test4 dbus-broker-launch[685]: Looking up NSS user entry for 'pulse'... Jul 04 13:09:52 VM-Test4 dbus-broker-launch[685]: NSS returned no entry for 'pulse' Jul 04 13:09:52 VM-Test4 dbus-broker-launch[685]: Invalid user-name in /usr/share/dbus-1/system.d/pulseaudio-system.conf +27: user="pulse" Jul 04 13:09:52 VM-Test4 dbus-broker-launch[685]: Noticed file-system modification, trigger reload. Jul 04 13:09:52 VM-Test4 dbus-broker-launch[685]: Looking up NSS user entry for 'pulse'... Jul 04 13:09:52 VM-Test4 dbus-broker-launch[685]: NSS returned no entry for 'pulse' Jul 04 13:09:52 VM-Test4 dbus-broker-launch[685]: Invalid user-name in /usr/share/dbus-1/system.d/pulseaudio-system.conf +27: user="pulse" Jul 04 13:11:12 VM-Test4 systemd[1]: dbus-broker.service: Failed to spawn executor: Argument list too long Jul 04 13:11:12 VM-Test4 systemd[1]: dbus-broker.service: Failed to spawn 'reload' task: Argument list too long Jul 04 13:11:12 VM-Test4 systemd[1]: Reload failed for D-Bus System Message Bus. [VM-Test4 test_user]# ```

It seems that systemd updating from version 255.7-1 to 256 broke support for kernels 5.4 and, more rarely, 6.1.0 (on Arch Linux). By following the instructions to downgrade systemd, I have salvaged my school computer, where formatting and reinstalling Linux could not.
The only reason I found this was a search based on the one warning at image-making time that was different in 5.4 compared to other kernels, relating to missing DRM in platform/drivers. Seeing how obscure and painstaking it was for me to find, I thought I’d sign up and share.

Good luck

soundofthunder · 24 2024 13:47

Hi @Ktastrophe and welcome to the Manjaro community.

Thanks for your comments. I have moved this thread to the Feedback category, which seems more appropriate.

As a new user, please take some time to familiarise yourself with Forum requirements; in particular, the many ways to use the forum to your benefit. To that end, some or all these links will be invaluable:

Last, but not least, the Stable Update Announcements, which you should check frequently for important update related information.

Cheers.

Ktastrophe · 11 2024 19:08

Just checked up on the progress of this, since a lot of the chatter on GitHub seems to indicate it’s a case closed…
As of systemd v.256.5-1, my system is still failing to boot on kernel 5.4. I had to forcibly downgrade systemd again.
Thought someone should know!

soundofthunder · 12 2024 07:53

Has it not crossed your mind that updating your kernel might be a valid workaround for you?

Perhaps either of the current LTS kernels – 6.6.x – 6.1.x – might produce favourable results.

Manjaro Kernels

MrLavender · 12 2024 13:29

It is closed in systemd 256.6 which was released two days ago. That version is only available in Manjaro unstable branch at the moment.

nikgnomic · 14 2024 09:54

process-util: handle pidfd_spawn() returning E2BIG by kkourt · Pull Request #34005 · systemd/systemd · GitHub

Originally, this issue was reported in #1077204 - systemd fails to boot on a 5.4 kernel - Debian Bug report logs.

Fixes #33613

Systemd v256.1 is now available from Unstable and Testing branches

Branch compare for Manjaro - systemd