Extremely slow boot times due to run-credentials-systemd\x2dtmpfiles\x2dsetup.service.mount

I’m having very slow boot times, like Windows-slow or worse. It’s taking 1m32s to boot. I have a “decent” laptop Dell XPS 13 with 16Gb of ram.

This is the result of systemd-analyze critical-chain:

The time when unit became active or started is printed after the “@” character.
The time the unit took to start is printed after the “+” character.

graphical.target @1min 31.631s
└─sddm.service @1min 31.630s
  └─plymouth-quit.service @1min 31.496s +93ms
    └─systemd-user-sessions.service @1min 31.456s +36ms
      └─network.target @1min 31.453s
        └─NetworkManager.service @1min 31.423s +29ms
          └─dbus.service @1min 31.345s +44ms
            └─basic.target @1min 31.340s
              └─sockets.target @1min 31.340s
                └─dbus.socket @1min 31.340s
                  └─sysinit.target @1min 31.337s
                    └─systemd-timesyncd.service @1min 31.293s +42ms
                      └─systemd-tmpfiles-setup.service @1min 31.113s +163ms
                        └─local-fs.target @1min 31.109s
                          └─run-credentials-systemd\x2dtmpfiles\x2dsetup.service.mount @1min 31.117s
                            └─local-fs-pre.target @1.351s
                              └─systemd-tmpfiles-setup-dev.service @1.148s +201ms
                                └─kmod-static-nodes.service @857ms +39ms
                                  └─systemd-journald.socket @851ms
                                    └─-.mount @816ms
                                      └─-.slice @816ms

There is a gap between local-fs-pre.target and the next task of about 1m30s. What the heck is going on there?

Can you run the below command and and paste the output?

sudo systemctl status run-credentials-systemd\x2dtmpfiles\x2dsetup.service.mount

That doesn’t tell us much about your computer, see below

● run-credentials-systemd\x2dtmpfiles\x2dsetup.service.mount - /run/credentials/systemd-tmpfiles-setup.service
     Loaded: loaded (/proc/self/mountinfo)
     Active: active (mounted) since Mon 2023-03-27 19:36:16 CDT; 31min ago
      Until: Mon 2023-03-27 19:36:16 CDT; 31min ago
      Where: /run/credentials/systemd-tmpfiles-setup.service
       What: ramfs

Regarding the spec of my computer, this is the output:

System:
  Kernel: 6.1.19-1-MANJARO arch: x86_64 bits: 64 compiler: gcc v: 12.2.1
    parameters: BOOT_IMAGE=/@/boot/vmlinuz-6.1-x86_64
    root=UUID=f3ece844-9e18-4d9e-b340-209417be041f rw rootflags=subvol=@ quiet
    splash resume=UUID=759061c4-1440-4043-9212-44dc140b978a
    udev.log_priority=3
  Desktop: KDE Plasma v: 5.26.5 tk: Qt v: 5.15.8 wm: kwin_x11 vt: 1 dm: SDDM
    Distro: Manjaro Linux base: Arch Linux
Machine:
  Type: Laptop System: Dell product: XPS 13 9370 v: N/A
    serial: <superuser required> Chassis: type: 10 serial: <superuser required>
  Mobo: Dell model: 0F6P3V v: A00 serial: <superuser required> UEFI: Dell
    v: 1.21.0 date: 07/06/2022
Battery:
  ID-1: BAT0 charge: 26.8 Wh (57.5%) condition: 46.6/52.0 Wh (89.7%) volts: 8.3
    min: 7.6 model: LGC-LGC6.73 DELL H754V79 type: Li-poly serial: <filter>
    status: charging
  Device-1: hidpp_battery_0 model: Logitech Wireless Mouse M325
    serial: <filter> charge: 55% (should be ignored) rechargeable: yes
    status: discharging
Memory:
  RAM: total: 15.35 GiB used: 4.67 GiB (30.5%)
  RAM Report: permissions: Unable to run dmidecode. Root privileges required.
CPU:
  Info: model: Intel Core i7-8550U bits: 64 type: MT MCP arch: Coffee Lake
    gen: core 8 level: v3 note: check built: 2017 process: Intel 14nm family: 6
    model-id: 0x8E (142) stepping: 0xA (10) microcode: 0xF0
  Topology: cpus: 1x cores: 4 tpc: 2 threads: 8 smt: enabled cache:
    L1: 256 KiB desc: d-4x32 KiB; i-4x32 KiB L2: 1024 KiB desc: 4x256 KiB
    L3: 8 MiB desc: 1x8 MiB
  Speed (MHz): avg: 1838 high: 2000 min/max: 400/4000 scaling:
    driver: intel_pstate governor: powersave cores: 1: 2000 2: 2000 3: 2000
    4: 2000 5: 711 6: 2000 7: 2000 8: 2000 bogomips: 32012
  Flags: 3dnowprefetch abm acpi adx aes aperfmperf apic arat
    arch_capabilities arch_perfmon art avx avx2 bmi1 bmi2 bts clflush
    clflushopt cmov constant_tsc cpuid cpuid_fault cx16 cx8 de ds_cpl dtes64
    dtherm dts epb ept ept_ad erms est f16c flexpriority flush_l1d fma fpu
    fsgsbase fxsr ht hwp hwp_act_window hwp_epp hwp_notify ibpb ibrs ida
    intel_pt invpcid invpcid_single lahf_lm lm mca mce md_clear mmx monitor
    movbe mpx msr mtrr nonstop_tsc nopl nx pae pat pbe pcid pclmulqdq pdcm
    pdpe1gb pebs pge pln pni popcnt pse pse36 pti pts rdrand rdseed rdtscp
    rep_good sdbg sep sgx smap smep ss ssbd sse sse2 sse4_1 sse4_2 ssse3 stibp
    syscall tm tm2 tpr_shadow tsc tsc_adjust tsc_deadline_timer vme vmx vnmi
    vpid x2apic xgetbv1 xsave xsavec xsaveopt xsaves xtopology xtpr
  Vulnerabilities:
  Type: itlb_multihit status: KVM: VMX disabled
  Type: l1tf mitigation: PTE Inversion; VMX: conditional cache flushes, SMT
    vulnerable
  Type: mds mitigation: Clear CPU buffers; SMT vulnerable
  Type: meltdown mitigation: PTI
  Type: mmio_stale_data mitigation: Clear CPU buffers; SMT vulnerable
  Type: retbleed mitigation: IBRS
  Type: spec_store_bypass mitigation: Speculative Store Bypass disabled via
    prctl
  Type: spectre_v1 mitigation: usercopy/swapgs barriers and __user pointer
    sanitization
  Type: spectre_v2 mitigation: IBRS, IBPB: conditional, STIBP: conditional,
    RSB filling, PBRSB-eIBRS: Not affected
  Type: srbds mitigation: Microcode
  Type: tsx_async_abort status: Not affected
Graphics:
  Device-1: Intel UHD Graphics 620 vendor: Dell driver: i915 v: kernel
    arch: Gen-9.5 process: Intel 14nm built: 2016-20 ports: active: DP-2,DP-5
    off: eDP-1 empty: DP-1,DP-3,DP-4 bus-ID: 00:02.0 chip-ID: 8086:5917
    class-ID: 0300
  Device-2: Realtek Integrated_Webcam_HD type: USB driver: uvcvideo
    bus-ID: 1-5:2 chip-ID: 0bda:58f4 class-ID: 0e02 serial: <filter>
  Device-3: Logitech C920 HD Pro Webcam type: USB
    driver: snd-usb-audio,uvcvideo bus-ID: 3-1.4.2:9 chip-ID: 046d:0892
    class-ID: 0102 serial: <filter>
  Display: x11 server: X.Org v: 21.1.7 compositor: kwin_x11 driver: X:
    loaded: modesetting alternate: fbdev,vesa dri: iris gpu: i915 display-ID: :0
    screens: 1
  Screen-1: 0 s-res: 3840x1080 s-dpi: 96 s-size: 1015x285mm (39.96x11.22")
    s-diag: 1054mm (41.51")
  Monitor-1: not-matched mapped: DP-1-2-1 pos: primary,right res: 1920x1080
    hz: 60 dpi: 96 size: 509x286mm (20.04x11.26") diag: 584mm (22.99") modes: N/A
  Monitor-2: not-matched mapped: DP-2 pos: left res: 1920x1080 hz: 60 dpi: 96
    size: 509x286mm (20.04x11.26") diag: 584mm (22.99") modes: N/A
  Monitor-3: eDP-1 note: disabled model: Sharp 0x148b built: 2017
    res: 1920x1080 dpi: 332 gamma: 1.2 size: 294x165mm (11.57x6.5")
    diag: 337mm (13.3") ratio: 16:9 modes: 3840x2160
  API: OpenGL v: 4.6 Mesa 22.3.5 renderer: Mesa Intel UHD Graphics 620 (KBL
    GT2) direct-render: Yes
Audio:
  Device-1: Intel Sunrise Point-LP HD Audio vendor: Dell driver: snd_hda_intel
    bus-ID: 3-1.3.1:6 v: kernel chip-ID: 0bda:48f0
    alternate: snd_soc_skl,snd_soc_avs,snd_sof_pci_intel_skl class-ID: 0300
    bus-ID: 00:1f.3 chip-ID: 8086:9d71 class-ID: 0403
  Device-2: Realtek USB Audio type: USB driver: hid-generic,usbhid
  Device-3: Logitech C920 HD Pro Webcam type: USB
    driver: snd-usb-audio,uvcvideo bus-ID: 3-1.4.2:9 chip-ID: 046d:0892
    class-ID: 0102 serial: <filter>
  Device-4: GN Netcom Jabra SPEAK 410 USB type: USB
    driver: jabra,snd-usb-audio,usbhid bus-ID: 3-1.4.3:10 chip-ID: 0b0e:0412
    class-ID: 0300 serial: <filter>
  Sound API: ALSA v: k6.1.19-1-MANJARO running: yes
  Sound Server-1: JACK v: 1.9.22 running: no
  Sound Server-2: PulseAudio v: 16.1 running: yes
  Sound Server-3: PipeWire v: 0.3.65 running: yes
Network:
  Device-1: Qualcomm Atheros QCA6174 802.11ac Wireless Network Adapter
    vendor: Rivet Networks Killer 1435 Wireless-AC driver: ath10k_pci v: kernel
    pcie: gen: 1 speed: 2.5 GT/s lanes: 1 bus-ID: 02:00.0 chip-ID: 168c:003e
    class-ID: 0280 temp: 48.0 C
  IF: wlp2s0 state: up mac: <filter>
  IP v4: <filter> type: dynamic noprefixroute scope: global
    broadcast: <filter>
  IP v6: <filter> type: dynamic noprefixroute scope: global
  IP v6: <filter> type: dynamic noprefixroute scope: global
  IP v6: <filter> type: noprefixroute scope: link
  Device-2: Realtek RTL8153 Gigabit Ethernet Adapter type: USB driver: r8152
    bus-ID: 4-1.3.4:6 chip-ID: 0bda:8153 class-ID: 0000 serial: <filter>
  IF: enp60s0u1u3u4 state: down mac: <filter>
  WAN IP: <filter>
Bluetooth:
  Device-1: Foxconn / Hon Hai type: USB driver: btusb v: 0.8 bus-ID: 1-7:3
    chip-ID: 0489:e0a2 class-ID: e001
  Report: rfkill ID: hci0 rfk-id: 0 state: up address: see --recommends
Logical:
  Message: No logical block device data found.
RAID:
  Message: No RAID data found.
Drives:
  Local Storage: total: 238.47 GiB used: 51.48 GiB (21.6%)
  SMART Message: Unable to run smartctl. Root privileges required.
  ID-1: /dev/nvme0n1 maj-min: 259:0 vendor: Toshiba
    model: KXG50ZNV256G NVMe 256GB size: 238.47 GiB block-size: physical: 512 B
    logical: 512 B speed: 31.6 Gb/s lanes: 4 type: SSD serial: <filter>
    rev: AADA4106 temp: 46.9 C scheme: GPT
  Message: No optical or floppy data found.
Partition:
  ID-1: / raw-size: 11.94 GiB size: 31.47 GiB (263.60%) used: 17.63 GiB (56.0%)
    fs: btrfs dev: /dev/nvme0n1p9 maj-min: 259:9 label: N/A
    uuid: f3ece844-9e18-4d9e-b340-209417be041f
  ID-2: /boot/efi raw-size: 512 MiB size: 511 MiB (99.80%)
    used: 612 KiB (0.1%) fs: vfat dev: /dev/nvme0n1p7 maj-min: 259:7 label: N/A
    uuid: F6FE-F1E2
  ID-3: /home raw-size: 2.93 GiB size: 63.88 GiB (2180.47%)
    used: 33.85 GiB (53.0%) fs: btrfs dev: /dev/nvme0n1p10 maj-min: 259:10
    label: N/A uuid: 22cbf2bb-d546-416d-a73d-0cb1b36df599
  ID-4: /var/cache raw-size: 11.94 GiB size: 31.47 GiB (263.60%)
    used: 17.63 GiB (56.0%) fs: btrfs dev: /dev/nvme0n1p9 maj-min: 259:9
    label: N/A uuid: f3ece844-9e18-4d9e-b340-209417be041f
  ID-5: /var/log raw-size: 11.94 GiB size: 31.47 GiB (263.60%)
    used: 17.63 GiB (56.0%) fs: btrfs dev: /dev/nvme0n1p9 maj-min: 259:9
    label: N/A uuid: f3ece844-9e18-4d9e-b340-209417be041f
Swap:
  Alert: No swap data was found.
Unmounted:
  ID-1: /dev/nvme0n1p1 maj-min: 259:1 size: 650 MiB fs: vfat label: ESP
    uuid: 9071-738F
  ID-2: /dev/nvme0n1p11 maj-min: 259:11 size: 26.72 GiB fs: btrfs label: N/A
    uuid: 22cbf2bb-d546-416d-a73d-0cb1b36df599
  ID-3: /dev/nvme0n1p12 maj-min: 259:12 size: 19.53 GiB fs: btrfs label: N/A
    uuid: f3ece844-9e18-4d9e-b340-209417be041f
  ID-4: /dev/nvme0n1p13 maj-min: 259:13 size: 34.23 GiB fs: btrfs label: N/A
    uuid: 22cbf2bb-d546-416d-a73d-0cb1b36df599
  ID-5: /dev/nvme0n1p2 maj-min: 259:2 size: 128 MiB fs: <superuser required>
    label: N/A uuid: N/A
  ID-6: /dev/nvme0n1p3 maj-min: 259:3 size: 126.5 GiB fs: ntfs label: OS
    uuid: BE6ACFE16ACF9515
  ID-7: /dev/nvme0n1p4 maj-min: 259:4 size: 990 MiB fs: ntfs label: N/A
    uuid: BED62CBCD62C7735
  ID-8: /dev/nvme0n1p5 maj-min: 259:5 size: 11.32 GiB fs: ntfs label: Image
    uuid: 548641C28641A578
  ID-9: /dev/nvme0n1p6 maj-min: 259:6 size: 1.08 GiB fs: ntfs
    label: DELLSUPPORT uuid: E64444CC4444A0E3
  ID-10: /dev/nvme0n1p8 maj-min: 259:8 size: 2 GiB fs: swap label: N/A
    uuid: 2dd963d5-9874-468d-8f5f-cdca301806f3
USB:
  Hub-1: 1-0:1 info: Hi-speed hub with single TT ports: 12 rev: 2.0
    speed: 480 Mb/s chip-ID: 1d6b:0002 class-ID: 0900
  Device-1: 1-5:2 info: Realtek Integrated_Webcam_HD type: Video
    driver: uvcvideo interfaces: 4 rev: 2.0 speed: 480 Mb/s power: 500mA
    chip-ID: 0bda:58f4 class-ID: 0e02 serial: <filter>
  Device-2: 1-7:3 info: Foxconn / Hon Hai type: Bluetooth driver: btusb
    interfaces: 2 rev: 1.1 speed: 12 Mb/s power: 100mA chip-ID: 0489:e0a2
    class-ID: e001
  Device-3: 1-10:4 info: Shenzhen Goodix Fingerprint Reader
    type: Abstract (modem),CDC-Data driver: cdc_acm interfaces: 2 rev: 2.0
    speed: 12 Mb/s power: 100mA chip-ID: 27c6:5385 class-ID: 0a00
    serial: <filter>
  Hub-2: 2-0:1 info: Super-speed hub ports: 6 rev: 3.0 speed: 5 Gb/s
    chip-ID: 1d6b:0003 class-ID: 0900
  Hub-3: 3-0:1 info: Hi-speed hub with single TT ports: 2 rev: 2.0
    speed: 480 Mb/s chip-ID: 1d6b:0002 class-ID: 0900
  Hub-4: 3-1:2 info: Genesys Logic Hub ports: 4 rev: 2.1 speed: 480 Mb/s
    chip-ID: 05e3:0610 class-ID: 0900
  Device-1: 3-1.2:3 info: Google Nexus/Pixel Device (MTP) type: Still Imaging
    driver: N/A interfaces: 1 rev: 2.0 speed: 480 Mb/s power: 500mA
    chip-ID: 18d1:4ee1 class-ID: 0601 serial: <filter>
  Hub-5: 3-1.3:4 info: Genesys Logic Hub ports: 4 rev: 2.1 speed: 480 Mb/s
    chip-ID: 05e3:0612 class-ID: 0900
  Device-1: 3-1.3.1:6 info: Realtek USB Audio type: HID
    driver: hid-generic,usbhid interfaces: 1 rev: 2.0 speed: 480 Mb/s
    power: 100mA chip-ID: 0bda:48f0 class-ID: 0300
  Device-2: 3-1.3.3:8 info: Apple Aluminium Keyboard (ANSI)
    type: Keyboard,Mouse driver: apple,usbhid interfaces: 2 rev: 2.0
    speed: 12 Mb/s power: 100mA chip-ID: 05ac:024f class-ID: 0301
  Hub-6: 3-1.4:5 info: Genesys Logic USB2.1 Hub ports: 4 rev: 2.1
    speed: 480 Mb/s chip-ID: 05e3:0611 class-ID: 0900
  Device-1: 3-1.4.1:7 info: Logitech Unifying Receiver type: Mouse,HID
    driver: logitech-djreceiver,usbhid interfaces: 2 rev: 2.0 speed: 12 Mb/s
    power: 98mA chip-ID: 046d:c52f class-ID: 0300
  Device-2: 3-1.4.2:9 info: Logitech C920 HD Pro Webcam type: Video,Audio
    driver: snd-usb-audio,uvcvideo interfaces: 4 rev: 2.0 speed: 480 Mb/s
    power: 500mA chip-ID: 046d:0892 class-ID: 0102 serial: <filter>
  Device-3: 3-1.4.3:10 info: GN Netcom Jabra SPEAK 410 USB type: Audio,HID
    driver: jabra,snd-usb-audio,usbhid interfaces: 4 rev: 2.0 speed: 12 Mb/s
    power: 500mA chip-ID: 0b0e:0412 class-ID: 0300 serial: <filter>
  Hub-7: 4-0:1 info: Super-speed hub ports: 2 rev: 3.1 speed: 10 Gb/s
    chip-ID: 1d6b:0003 class-ID: 0900
  Hub-8: 4-1:2 info: Genesys Logic USB3.2 Hub ports: 4 rev: 3.2
    speed: 10 Gb/s chip-ID: 05e3:0625 class-ID: 0900
  Hub-9: 4-1.3:3 info: Genesys Logic USB3.2 Hub ports: 4 rev: 3.2
    speed: 5 Gb/s chip-ID: 05e3:0621 class-ID: 0900
  Device-1: 4-1.3.2:5 info: Genesys Logic USB Storage type: Mass Storage
    driver: usb-storage interfaces: 1 rev: 3.2 speed: 5 Gb/s power: 896mA
    chip-ID: 05e3:0754 class-ID: 0806 serial: <filter>
  Device-2: 4-1.3.4:6 info: Realtek RTL8153 Gigabit Ethernet Adapter
    type: Network driver: r8152 interfaces: 1 rev: 3.0 speed: 5 Gb/s power: 288mA
    chip-ID: 0bda:8153 class-ID: 0000 serial: <filter>
  Hub-10: 4-1.4:4 info: Genesys Logic GL3523 Hub ports: 4 rev: 3.2
    speed: 5 Gb/s chip-ID: 05e3:0620 class-ID: 0900
Sensors:
  System Temperatures: cpu: 50.0 C pch: 46.0 C mobo: 42.0 C sodimm: SODIMM C
  Fan Speeds (RPM): cpu: 4703
Info:
  Processes: 286 Uptime: 34m wakeups: 12 Init: systemd v: 252
  default: graphical tool: systemctl Compilers: gcc: 12.2.1 clang: 15.0.7
  Packages: pm: pacman pkgs: 1349 libs: 423 tools: pamac,yay pm: flatpak
  pkgs: 0 Shell: Zsh v: 5.9 default: Bash v: 5.1.16 running-in: konsole
  inxi: 3.3.25

Can you paste the contents of /etc/fstab?

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a device; this may
# be used with UUID= as a more robust way to name devices that works even if
# disks are added and removed. See fstab(5).
#
# <file system>             <mount point>  <type>  <options>  <dump>  <pass>
UUID=F6FE-F1E2                              /boot/efi    vfat    umask=0077                         0 2 
UUID=759061c4-1440-4043-9212-44dc140b978a   swap         swap    noatime                            0 0 
UUID=f3ece844-9e18-4d9e-b340-209417be041f   /            btrfs   subvol=/@,discard=async,ssd        0 0 
UUID=f3ece844-9e18-4d9e-b340-209417be041f   /var/cache   btrfs   subvol=/@cache,discard=async,ssd   0 0 
UUID=f3ece844-9e18-4d9e-b340-209417be041f   /var/log     btrfs   subvol=/@log,discard=async,ssd     0 0 
UUID=22cbf2bb-d546-416d-a73d-0cb1b36df599   /home        btrfs   discard=async,ssd                  0 0 
tmpfs                                       /tmp         tmpfs   noatime,mode=1777                  0 0 
/dev/nvme0n1p3                              /mnt/win     ntfs    noatime,noauto,nodiratime,ro       0 0 

Can you paste the output of the following commands?

ls ~/.config/user-tmpfiles.d/
ls ~/.local/share/user-tmpfiles.d/
ls /usr/share/user-tmpfiles.d/

All three directories do not exist: “no such file or directory”

How about this?

for file in /tmp/*; do sudo du -sh "$file"; done
28K     /tmp/manjaro-hello-preferences.json
0       /tmp/plasma-csd-generator.WMuAVk
0       /tmp/pyright-4399-dLDbSHChfj1t
220K    /tmp/pyright-4399-ksAOCv46Ncb0
0       /tmp/python-languageserver-cancellation
0       /tmp/qipc_sharedmemory_bdedbfdbabcbafb56cd51926635d136055035e4dd9c7d1f5003af98
0       /tmp/qipc_sharedmemory_MSMNotifierforPlasmad7cd3d9ed002d00038761dcd548b1461629c9f02
0       /tmp/qipc_systemsem_bdedbfdbabcbafb56cd51926635d136055035e4dd9c7d1f5003af98
0       /tmp/qipc_systemsem_bfcabbfaecebac69dadd18f634141b115a30ce0998c7a1e61e4cfb
0       /tmp/qipc_systemsem_MSMNotifierforPlasmad7cd3d9ed002d00038761dcd548b1461629c9f02
0       /tmp/sddm-:0-QVsSOm
0       /tmp/sddm-auth0c664e81-ffaf-4fb3-ac3b-6dd78ca9247c
0       /tmp/systemd-private-c9c6ce5e7cc04c96ae31bf1c3b1714cf-bluetooth.service-PZK0OT
0       /tmp/systemd-private-c9c6ce5e7cc04c96ae31bf1c3b1714cf-bolt.service-eOKdyO
0       /tmp/systemd-private-c9c6ce5e7cc04c96ae31bf1c3b1714cf-ModemManager.service-slLLzz
0       /tmp/systemd-private-c9c6ce5e7cc04c96ae31bf1c3b1714cf-systemd-logind.service-7uODMF
0       /tmp/systemd-private-c9c6ce5e7cc04c96ae31bf1c3b1714cf-systemd-timesyncd.service-ew5W2w
0       /tmp/systemd-private-c9c6ce5e7cc04c96ae31bf1c3b1714cf-upower.service-gnjkyY
0       /tmp/Temp-dedac32f-78ce-44d9-aff4-4aff71465c72

Removing this didn´t have any effect, so I put it back.

Sorry for the wild goose chase, but we gotta keep looking at things until we find a clue. Next idea: after a fresh boot, run the following

sudo journalctl -b 0 | grep -E "mount|tmp|tmp.mount"
Mar 28 16:12:06 xxxxx kernel: devtmpfs: initialized
Mar 28 16:12:06 xxxxx systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point.
Mar 28 16:12:06 xxxxx systemd[1]: Starting Remount Root and Kernel File Systems...
Mar 28 16:12:06 xxxxx systemd[1]: Finished Remount Root and Kernel File Systems.
Mar 28 16:13:36 xxxxx systemd[1]: Mounting /tmp...
Mar 28 16:13:36 xxxxx systemd[1]: Mounted /tmp.
Mar 28 16:13:36 xxxxx systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 554 (systemd-binfmt)
Mar 28 16:13:36 xxxxx audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-tmpfiles-setup comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Mar 28 16:13:36 xxxxx kernel: audit: type=1130 audit(1680038016.355:13): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-tmpfiles-setup comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Mar 28 16:13:36 xxxxx audit[559]: SYSTEM_BOOT pid=559 uid=0 auid=4294967295 ses=4294967295 msg=' comm="systemd-update-utmp" exe="/usr/lib/systemd/systemd-update-utmp" hostname=? addr=? terminal=? res=success'
Mar 28 16:13:36 xxxxx kernel: audit: type=1127 audit(1680038016.398:15): pid=559 uid=0 auid=4294967295 ses=4294967295 msg=' comm="systemd-update-utmp" exe="/usr/lib/systemd/systemd-update-utmp" hostname=? addr=? terminal=? res=success'
Mar 28 16:13:36 xxxxx audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-update-utmp comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Mar 28 16:13:36 xxxxx kernel: audit: type=1130 audit(1680038016.445:16): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-update-utmp comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Mar 28 16:13:41 xxxxx dbus-daemon[564]: [system] Activating via systemd: service name='org.freedesktop.UDisks2' unit='udisks2.service' requested by ':1.19' (uid=964 pid=668 comm="/usr/bin/sddm-greeter --socket /tmp/sddm-:0-lRayAB")
Mar 28 16:13:41 xxxxx dbus-daemon[564]: [system] Activating via systemd: service name='org.freedesktop.UPower' unit='upower.service' requested by ':1.19' (uid=964 pid=668 comm="/usr/bin/sddm-greeter --socket /tmp/sddm-:0-lRayAB")
Mar 28 16:13:49 xxxxx dbus-daemon[564]: [system] Activating via systemd: service name='org.freedesktop.home1' unit='dbus-org.freedesktop.home1.service' requested by ':1.23' (uid=0 pid=719 comm="/usr/lib/sddm/sddm-helper --socket /tmp/sddm-auth9")
Mar 28 16:13:54 xxxxx manjaro-hello[1055]: DEBUG:root:json : /tmp/manjaro-hello-preferences.json
Mar 28 16:13:59 xxxxx systemd[1]: run-user-964.mount: Deactivated successfully.

Looking at dmesg there is a 1m30s gap where only audit entries appear. It seems like nothing is happening.

I think it has to be some driver waiting on a device or something like that.

Can you paste the audit messages you described? I have found one case where a similar issue had clues in the audit messages.

[   15.573524] ath: Country alpha2 being used: 00
[   15.573525] ath: Regpair used: 0x6c
[   15.578447] ath10k_pci 0000:02:00.0 wlp2s0: renamed from wlan0
[   15.937087] input: HDA Intel PCH Headphone Mic as /devices/pci0000:00/0000:00:1f.3/sound/card0/input28
[   15.937149] input: HDA Intel PCH HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:1f.3/sound/card0/input29
[   15.937202] input: HDA Intel PCH HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:1f.3/sound/card0/input30
[   15.937263] input: HDA Intel PCH HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:1f.3/sound/card0/input31
[   20.606327] audit: type=1131 audit(1680051812.582:10): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-rfkill comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  103.226609] audit: type=1130 audit(1680051895.202:11): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=plymouth-read-write comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  103.249990] audit: type=1130 audit(1680051895.226:12): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-binfmt comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  103.362829] audit: type=1130 audit(1680051895.339:13): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-tmpfiles-setup comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  103.363941] audit: type=1334 audit(1680051895.339:14): prog-id=29 op=LOAD
[  103.393264] audit: type=1127 audit(1680051895.366:15): pid=569 uid=0 auid=4294967295 ses=4294967295 msg=' comm="systemd-update-utmp" exe="/usr/lib/systemd/systemd-update-utmp" hostname=? addr=? terminal=? res=success'
[  103.425933] audit: type=1130 audit(1680051895.399:16): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-update-utmp comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  103.447310] audit: type=1130 audit(1680051895.419:17): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-timesyncd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  103.454204] audit: type=1130 audit(1680051895.426:18): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=cronie comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  103.458223] audit: type=1334 audit(1680051895.429:19): prog-id=30 op=LOAD
[  103.458346] audit: type=1334 audit(1680051895.429:20): prog-id=31 op=LOAD
[  103.491222] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[  103.491227] Bluetooth: BNEP filters: protocol multicast
[  103.491232] Bluetooth: BNEP socket layer initialized

To be completely honest, I am at a bit of a loss on this one. The unit credited with extending the boot times is all about the tmpfs, but I can find nothing out of the ordinary related to your tmpfs. There are also extremely few documented cases related to this unit, but I will link the ones I found below in case they are useful. It definitely shouldn’t be doing this, but I’m not really sure why it is.

Sorry, I hope someone else is able to help.

What’s strange is that a sudo systemd-analyze blame doesn´t show any task taking longer than 1s…

Another clue: the gap is exactly the defined timeout, assuming the timeout on yours is the same as mine:

systemctl show "run-credentials-systemd\x2dtmpfiles\x2dsetup.service.mount"

Look for

TimeoutUSec=1min 30s

2 Likes

Problem solved. The issue was that the UUID in fstab for the swap partition was wrong. At some point, a couple of weeks ago, I deleted and created again the swap partition to move it to another disk. I think fstab was never updated (I’m not sure if it supposed to be updated automatically)

1 Like

fstab is not updated automatically unless the program you used to change the swap partition has that as an explicit feature. I assume you just used swapoff and swapon, so it isn’t. Glad to hear you managed to figure this out!