Can not boot after upgraded

Support System Updates
1

I got these errors. I tried to install latest linux kernel (for example linux 612) but it didn’t work. I tried to search for solutions but can not find any related. I tried to add SYSTEMD_CGROUP_ENABLE_LEGACY_FORCE=1 and systemd.unified_cgroup_hierarchy=0 but didn’t work.

[  OK  ] Reached target Login Prompts.
[  OK  ] Starting Load udev Rules from Credentials...
[  OK  ] Reached target Network.
[  OK  ] Reached target Network is Online.
[  OK  ] Reached target Path Units.
[  OK  ] Reached target Local Encrypted Volumes.
[  OK  ] Reached target Emergency Shell.
[  OK  ] Starting tell Plymouth to Write Out Runtime Data...
         Starting Set Up Additional Binary Formats...
[FAILED] Failed to start Systemd Journal Service.
See 'systemctl status systemd-journal-service' for details.
[FAILED] Failed to start Tell Plymouth to Write Out Runtime Data.
See 'systemctl status Plymouth-write-out-runtime-data.service' for details.
[FAILED] Failed to start Set Up Additional Binary Formats.
See 'systemctl status systemd-binfmt-service' for details.
         Starting Rule-based Manager for Device Events and Files...
[FAILED] Failed to start Rule-based Manager for Device Events and Files.
See 'systemctl status systemd-udevd.service' for details.
         Starting Load udev Rules from Credentials...
[FAILED] Failed to start Load udev Rules from Credentials.
See 'systemctl status systemd-udev-load-credentials.service' for details.
         Starting Rule-based Manager for Device Events and Files...
[FAILED] Failed to start Rule-based Manager for Device Events and Files.
See 'systemctl status systemd-udevd.service' for details.
         Starting Load udev Rules from Credentials...
[FAILED] Failed to start Load udev Rules from Credentials.
See 'systemctl status systemd-udev-load-credentials.service' for details.
         Starting Rule-based Manager for Device Events and Files...
[FAILED] Failed to start Rule-based Manager for Device Events and Files.
See 'systemctl status systemd-udevd.service' for details.
         Starting Show Plymouth Boot Screen...
[FAILED] Failed to start Show Plymouth Boot Screen.
See 'systemctl status plymouth-start.service' for details.
[  OK  ] Started Forward Password Requests to Plymouth Directory Watch.
< 1 of 3 > A start job is running for /dev/disk/by-uuid/9347-E4CF (18s / 1min 30s)
[    ]

[   96.297765] systemd[1]: ldconfig.service: Releasing resources...
[   96.297827] systemd[1]: Child 845 (16) died (code=exited, status=1/FAILURE)
[   96.297870] systemd[1]: systemd-journal-catalog-update.service: Child 845 belongs to systemd-journal-catalog-update.service.
[   96.297876] systemd[1]: systemd-journal-catalog-update.service: Main process exited, code=exited, status=1/FAILURE
[   96.297909] systemd[1]: systemd-journal-catalog-update.service: Service will not restart (restarting)
[   96.297904] systemd[1]: systemd-journal-catalog-update.service: Changed start -> failed
[   96.297923] systemd[1]: systemd-journal-catalog-update.service: Job 66 systemd-journal-catalog-update.service/start finished, result=failed
[   96.2979240] systemd[1]: Failed to start Rebuild Journal Catalog.
See 'systemctl status ldconfig.service' for details.
[   96.305276] systemd[1]: systemd-journal-catalog-update.service: Consumed 1ms CPU time, 508K memory peak.
[   96.305308] systemd[1]: systemd-journal-catalog-update.service: Releasing resources...
[   96.307113] systemd[1]: systemd-update-done.service: ConditionNeedsUpdate=/var succeeded.
[   96.307135] systemd[1]: systemd-update-done.service: ConditionNeedsUpdate=/etc succeeded.
[   96.307165] systemd[1]: systemd-update-done.service: Will spawn child (service_enter_start): /usr/lib/systemd/systemd-update-done
See 'systemctl status systemd-journal-catalog-update.service' for details.
[   96.310863] systemd[1]: systemd-update-done.service: Passing 0 fds to service
[   96.310863] systemd[1]: systemd-update-done.service: about to execute: /usr/lib/systemd/systemd-update-done
[   96.311235] systemd[1]: Serializing sd-executor to memfd
[   96.312385] systemd[1]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy
[   96.313333] systemd[1]: systemd-update-done.service: Forked /usr/lib/systemd/systemd-update-done as 846 (via CLON
[   96.314489] systemd[1]: Closing set fd 50 (socket: [32795])
[   96.315052] systemd[1]: Closing set fd 49 (socket: [32791])
[   96.315659] systemd[1]: systemd-update-done.service: Changed dead -> start
[   96.316307] systemd[1]: Starting Update is Completed...
[   96.317146] systemd[1]: Received SIGCHLD from PID 846 (16).
[   96.317157] systemd[1]: Child 846 (16) died (code=exited, status=1/FAILURE)
[   96.317183] systemd[1]: systemd-update-done.service: Child 846 belongs to systemd-update-done.service.
[   96.317189] systemd[1]: systemd-update-done.service: Main process exited, code=exited, status=1/FAILURE
[   96.317295] systemd[1]: systemd-update-done.service: Service will not restart (restarting)
[   96.317323] systemd[1]: systemd-update-done.service: Changed start -> failed
[   96.317463] systemd[1]: systemd-update-done.service: Job 27 systemd-update-done.service/start finished, result=failed
[   96.317974] systemd[1]: Failed to start Update is Completed.
[FAILED] Failed to start Update is Completed.
[   96.320809] systemd[1]: Startup finished in 34.998s (firmware) + 29.962s (loader) + 5.491s (kernel) + 1min 30.839s (initrd) + 41.231s (userspace) = 2min 42.523s.
[   96.320925] systemd[1]: System is tainted: var-run-bad
[   96.320938] systemd[1]: systemd-update-done.service: Consumed 779us CPU time, 508K memory peak.
See 'systemctl status systemd-update-done.service' for details.
2

Please do not post photos of text - I have removed them

How did you come the conclusion of using the unconventional kernel arguments?

The log output makes no sense to me - is this caused by applying updates at shutdown or … ?

3

I saw this thread that have some similar errors.

So I tried the methods mentioned.

And there are more logs after A start job is running for

 [FAILED] Starting Rule-based Manager for Device Events and Files...
See 'systemctl status systemd-udevd.service' for details.
[FAILED] Failed to start Rule-based Manager for Device Events and Files.
See 'systemctl status systemd-udevd.service' for details.
         Starting Load udev Rules from Credentials...
[FAILED] Failed to start Load udev Rules from Credentials.
See 'systemctl status systemd-udev-load-credentials.service' for details.
[FAILED] Failed to start Rule-based Manager for Device Events and Files.
See 'systemctl status systemd-udevd.service' for details.
[FAILED] Failed to start Load udev Rules from Credentials.
See 'systemctl status systemd-udev-load-credentials.service' for details.
         Starting Show Plymouth Boot Screen...
See 'systemctl status plymouth-start.service' for details.
[  OK  ] Started Forward Password Requests to Plymouth Directory Watch.
[DEPEND] Dependency failed for /home.
[DEPEND] Dependency failed for /dev/disk/by-uuid/41c4974c-41a8-4855-9f8f-c367be00ceca.
[DEPEND] Dependency failed for /dev/disk/by-uuid/9347-E4CF.
[DEPEND] Dependency failed for /boot/efi.
[DEPEND] Dependency failed for /dev/disk/by-uuid/cbbe3036-2219-4bad-b524-f9016777a9c0.
[DEPEND] Dependency failed for Record System Boot/Shutdown in UTMP.
[DEPEND] Dependency failed for Load/Save OS Random Seed.
[DEPEND] Dependency failed for Virtual Machine and Container Storage (Compatibility).
[DEPEND] Dependency failed for TPM2 SRK Setup.
[DEPEND] Dependency failed for File System Check on /dev/disk/by-uuid/cbbe3036-2219-4bad-b524-f9016777a9c0.
[  OK  ] Reached target System Time Set.
[  OK  ] Listening on D-Bus System Message Bus Socket.
[  OK  ] Reached target Socket Units.
         Starting Create System Files and Directories...
[FAILED] Failed to start Create System Files and Directories.
See 'systemctl status systemd-tmpfiles.service' for details.
         Starting Rebuild Dynamic Linker Cache...
[FAILED] Failed to start Rebuild Dynamic Linker Cache.
See 'systemctl status ldconfig.service' for details.
[FAILED] Failed to start Rebuild Journal Catalog.
See 'systemctl status systemd-journal-catalog-update.service' for details.
         Starting Update is Completed...
[FAILED] Failed to start Update is Completed.
See 'systemctl status systemd-update-done.service' for details.
4

@linux-aarhus Any idea?

5

I have no idea - only guesses

  • your fstab is malformed
  • you are using secure boot
    • your unified kernel is corrupt
  • you have an invalid rule

I have no clue - really - none

6

@linux-aarhus I had filesystem check and there is no error. I guess pacman upgrade wouldn’t touch fstab right? secure boot is not enabled.
Is there any way to go into Emergency Shell in this state? I tried to add emergency.target or rescue.target or 3 at the end of command line but I can find a way to enter any shell.
I think maybe enter a shell and run ‘systemctl status systemd-journal-service’ can provide more details.

7

I have no idea - sorry - I suggest starting with

→ [root tip] [recovery] Basic Manjaro Linux Recovery

8

Not sure why you’d want to do that.
The name implies something different than what you seem to expect.

To analyze and fix your system, you’ll likely have to boot an ISO - the Manjaro ISO is preferred.
Then chroot and … fix

9

@Nachlese I have a usb live CD available. I tried manjaro-chroot and install other kernel. But how to fix it?

10

You did?
How?

the kernel is likely not the problem

start with (from within chroot):
pacman-mirrors -f
pacman -Syu

@linux-aarhus linked you to a guide

11

manjaro live cd

pacman -Syu return no any updates.

12

from there:
manjaro-chroot -a

it may be what you did - but you didn’t say you did

How did you install the kernel?

mhwd-kernel -i ...
doesn’t work in chroot

13

I did this.

for example pacman -S linux612

14

refresh mirrors?
run update again?
(from within chroot …)

15

Like I said I have run pacman -Syu. And why should I change my mirror?

16

I didn’t say “change”.
I suggested to refresh.

From the start, it looked like some file corruption issue to me.
But that was just a hunch.

I do not know why basically everything would fail to load while the system is up to date and fine and dandy. :man_shrugging:

inxi -zv8

enough space left on the devices?
(just another guess)

17

checked. yes.

18

well - if we could see what, until now, only you can
we could form our own opinion

19

@Nachlese Sorry I can not run this command right now. I just tried @linux-aarhus good advice and now the chroot is completely destroyed.

[manjaro log]# pacman -Syu
:: Synchronizing package databases...
 core is up to date
 extra                                                                                                                                                                                                                              8.3 MiB  12.4 MiB/s 00:01 [##################################################################################################################################################################] 100%
 multilib is up to date
 sublime-text                                                                                                                                                                                                                    1865.0   B  2.06 KiB/s 00:01 [##################################################################################################################################################################] 100%
:: Starting full system upgrade...
 there is nothing to do
[manjaro ~]# pacman -Qqem
clash-for-windows-bin
cpufreqctl
debtap
glibc-eac
glibc-eac-locales
khotkeys
lib32-glibc-eac
lib32-libva-vdpau-driver
manjaro-hotfixes
manjaro-settings-manager-kcm
parsec
plasma-simplemenu
reiserfsprogs
rustdesk-bin
spectre-meltdown-checker
sunshine-bin
systemd-kcm
visual-studio-code-bin
visual-studio-code-insiders-bin
[manjaro ~]# pacman -Qqem > alien-package-list.txt
[manjaro ~]# ls
alien-package-list.txt  nohup.out
[manjaro ~]# pacman -Rdd < (pacman -Qqem)
bash: syntax error near unexpected token `('
[manjaro ~]# pacman -Rdd $(pacman -Qqem)
​
Packages (19) clash-for-windows-bin-0.20.39-3  cpufreqctl-10.1.2-3  debtap-3.5.1-1  glibc-eac-2.40-1  glibc-eac-locales-2.40-1  khotkeys-5.27.11-1  lib32-glibc-eac-2.40-1  lib32-libva-vdpau-driver-0.7.4-8  manjaro-hotfixes-2024.1-2  manjaro-settings-manager-kcm-0.5.7-21  parsec-150-1  plasma-simplemenu-1.0.12-1  reiserfsprogs-3.6.27-5  rustdesk-bin-1.3.1-3  spectre-meltdown-checker-0.46+23+g0f2edb1-1
              sunshine-bin-0.23.1-3  systemd-kcm-1.2.1-8  visual-studio-code-bin-1.94.2-1  visual-studio-code-insiders-bin-1727243173-1
​
Total Removed Size:  1398.61 MiB
​
:: Do you want to remove these packages? [Y/n] 
:: Running pre-transaction hooks...
(1/1) Removing old entries from the info directory file...
:: Processing package changes...
( 1/19) removing clash-for-windows-bin                                                                                                                                                                                                                        [##################################################################################################################################################################] 100%
( 2/19) removing cpufreqctl                                                                                                                                                                                                                                   [##################################################################################################################################################################] 100%
( 3/19) removing debtap                                                                                                                                                                                                                                       [##################################################################################################################################################################] 100%
==> /var/cache/debtap has not been removed
( 4/19) removing glibc-eac                                                                                                                                                                                                                                    [##################################################################################################################################################################] 100%
( 5/19) removing glibc-eac-locales                                                                                                                                                                                                                            [##################################################################################################################################################################] 100%
( 6/19) removing khotkeys                                                                                                                                                                                                                                     [##################################################################################################################################################################] 100%
call to execv failed (No such file or directory)
error: command failed to execute correctly
( 7/19) removing lib32-glibc-eac                                                                                                                                                                                                                              [##################################################################################################################################################################] 100%
( 8/19) removing lib32-libva-vdpau-driver                                                                                                                                                                                                                     [##################################################################################################################################################################] 100%
( 9/19) removing manjaro-hotfixes                                                                                                                                                                                                                             [##################################################################################################################################################################] 100%
(10/19) removing manjaro-settings-manager-kcm                                                                                                                                                                                                                 [##################################################################################################################################################################] 100%
(11/19) removing parsec                                                                                                                                                                                                                                       [##################################################################################################################################################################] 100%
(12/19) removing plasma-simplemenu                                                                                                                                                                                                                            [##################################################################################################################################################################] 100%
(13/19) removing reiserfsprogs                                                                                                                                                                                                                                [##################################################################################################################################################################] 100%
call to execv failed (No such file or directory)
error: command failed to execute correctly
(14/19) removing rustdesk-bin                                                                                                                                                                                                                                 [##################################################################################################################################################################] 100%
(15/19) removing spectre-meltdown-checker                                                                                                                                                                                                                     [##################################################################################################################################################################] 100%
(16/19) removing sunshine-bin                                                                                                                                                                                                                                 [##################################################################################################################################################################] 100%
(17/19) removing systemd-kcm                                                                                                                                                                                                                                  [##################################################################################################################################################################] 100%
(18/19) removing visual-studio-code-bin                                                                                                                                                                                                                       [##################################################################################################################################################################] 100%
(19/19) removing visual-studio-code-insiders-bin                                                                                                                                                                                                              [##################################################################################################################################################################] 100%
:: Running post-transaction hooks...
(1/9) Reloading system manager configuration...
call to execv failed (No such file or directory)
error: command failed to execute correctly
(2/9) Reloading user manager configuration...
call to execv failed (No such file or directory)
error: command failed to execute correctly
(3/9) Reloading device manager configuration...
call to execv failed (No such file or directory)
error: command failed to execute correctly
(4/9) Arming ConditionNeedsUpdate...
call to execv failed (No such file or directory)
error: command failed to execute correctly
(5/9) Updating the MIME type database...
call to execv failed (No such file or directory)
error: command failed to execute correctly
(6/9) Refreshing PackageKit...
call to execv failed (No such file or directory)
error: command failed to execute correctly
(7/9) Reloading system bus configuration...
call to execv failed (No such file or directory)
error: command failed to execute correctly
(8/9) Updating icon theme caches...
call to execv failed (No such file or directory)
error: command failed to execute correctly
(9/9) Updating the desktop file MIME type cache...
call to execv failed (No such file or directory)
error: command failed to execute correctly
[manjaro ~]# pacman -Syu
bash: /usr/bin/pacman: cannot execute: required file not found
[manjaro ~]# exit
exit
 --> umount: [/mnt/home]
 --> umount: [/mnt/var]
 --> umount: [/mnt/boot/efi]
 --> umount: [/mnt]
    ~  manjaro-chroot -a                                                                                                                                                                                                                                                                                                                                                                                       127 ✘  21m 39s  
==> Mounting (ManjaroLinux) [/dev/nvme0n1p1]
 --> mount: [/mnt]
 --> mount: [/mnt/boot/efi]
 --> mount: [/mnt/var]
 --> mount: [/mnt/home]
chroot: failed to run command '/bin/bash': No such file or directory
 --> umount: [/mnt/home]
 --> umount: [/mnt/var]
 --> umount: [/mnt/boot/efi]
 --> umount: [/mnt]
20

mkay - altough I don’t see why that would not be possible

From what you posted that conclusion does not follow.
But:
whatever we can’t see (but only you can)
we must trust your story.