Systemd-240: unexpected udevd errors

raid
systemd
udev
mdadm
#1

[Splitting from my post, here]

Since installing systemd-240 (currently 240.1-7 installed), I’ve been getting these errors:

$ journalctl -p 3 -b 0
Dec 26 10:28:11 Jammin1 systemd-udevd[443]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sda6 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[426]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sdb3 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[387]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sdb5 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[420]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sda2 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[407]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sdb4 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[392]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sdb2 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[406]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sda5 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[437]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sdb6 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[440]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sda4 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[403]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sda3 --offroot ${DEVLINKS}': Input/output error

And here are the journal entries, along with the corresponding dmesg entries, in chronological order (note, no udev messages in dmesg):

Journal and dmesg

Dec 26 10:28:11 Jammin1 kernel: sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/466 GiB)
Dec 26 10:28:11 Jammin1 kernel: sd 0:0:0:0: [sda] Write Protect is off
Dec 26 10:28:11 Jammin1 kernel: sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
Dec 26 10:28:11 Jammin1 kernel: sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Dec 26 10:28:11 Jammin1 kernel: sd 2:0:0:0: [sdb] 976773168 512-byte logical blocks: (500 GB/466 GiB)
Dec 26 10:28:11 Jammin1 kernel: sd 2:0:0:0: [sdb] Write Protect is off
Dec 26 10:28:11 Jammin1 kernel: sd 2:0:0:0: [sdb] Mode Sense: 00 3a 00 00
Dec 26 10:28:11 Jammin1 kernel: sd 2:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Dec 26 10:28:11 Jammin1 kernel:  sda: sda1 sda2 sda3 sda4 sda5 sda6 sda7
Dec 26 10:28:11 Jammin1 kernel:  sdb: sdb1 sdb2 sdb3 sdb4 sdb5 sdb6 sdb7
Dec 26 10:28:11 Jammin1 kernel: sd 0:0:0:0: [sda] supports TCG Opal
Dec 26 10:28:11 Jammin1 kernel: sd 0:0:0:0: [sda] Attached SCSI disk
Dec 26 10:28:11 Jammin1 kernel: sd 2:0:0:0: [sdb] supports TCG Opal
Dec 26 10:28:11 Jammin1 kernel: sd 2:0:0:0: [sdb] Attached SCSI disk

Dec 26 10:28:11 Jammin1 kernel: md/raid1:md4: active with 2 out of 2 mirrors
Dec 26 10:28:11 Jammin1 kernel: md4: detected capacity change from 0 to 12876513280
Dec 26 10:28:11 Jammin1 kernel: md/raid1:md3: active with 2 out of 2 mirrors
Dec 26 10:28:11 Jammin1 kernel: md3: detected capacity change from 0 to 32195477504
Dec 26 10:28:11 Jammin1 kernel: md/raid1:md0: active with 2 out of 2 mirrors
Dec 26 10:28:11 Jammin1 kernel: md0: detected capacity change from 0 to 32212254720
Dec 26 10:28:11 Jammin1 kernel: md/raid1:md1: active with 2 out of 2 mirrors
Dec 26 10:28:11 Jammin1 kernel: md1: detected capacity change from 0 to 5364514816
Dec 26 10:28:11 Jammin1 kernel: md/raid1:md2: active with 2 out of 2 mirrors
Dec 26 10:28:11 Jammin1 kernel: md2: detected capacity change from 0 to 21458059264
 
...
 
Dec 26 10:28:11 Jammin1 systemd[1]: Started udev Kernel Device Manager.
Dec 26 10:28:11 Jammin1 systemd-udevd[411]: Using default interface naming scheme 'v240'.
Dec 26 10:28:11 Jammin1 systemd-udevd[411]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Dec 26 10:28:11 Jammin1 systemd-udevd[401]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Dec 26 10:28:11 Jammin1 systemd-udevd[433]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sda4 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[404]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sdb3 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[403]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sdb6 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[400]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sdb4 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[384]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sda6 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[432]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sda2 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[431]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sdb5 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[422]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sda5 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[389]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sda3 --offroot ${DEVLINKS}': Input/output error
Dec 26 10:28:11 Jammin1 systemd-udevd[425]: Failed to wait spawned command '/usr/bin/mdadm --incremental --export /dev/sdb2 --offroot ${DEVLINKS}': Input/output error
 
** The next 4 lines correspond to the UUIDs of my RAID arrays.
 
Dec 26 10:28:12 Jammin1 systemd[1]: Found device /dev/disk/by-uuid/2d231bd8-05eb-4ead-aed5-75f8807cd2f7.
Dec 26 10:28:12 Jammin1 systemd[1]: Found device /dev/disk/by-uuid/4100d35b-a028-4c46-b94f-8217fd38ff68.
Dec 26 10:28:12 Jammin1 systemd[1]: Found device /dev/disk/by-uuid/7b19f59c-f76a-4f1e-b879-0de36bc1cdb2.
Dec 26 10:28:12 Jammin1 systemd[1]: Found device /dev/disk/by-uuid/0be80a87-eb88-4497-a5f7-f90a89944762.
Dec 26 10:28:12 Jammin1 systemd[1]: Condition check resulted in Set Up Additional Binary Formats being skipped.
Dec 26 10:28:12 Jammin1 systemd[1]: Condition check resulted in First Boot Wizard being skipped.
Dec 26 10:28:12 Jammin1 systemd[1]: Condition check resulted in File System Check on Root Device being skipped.
Dec 26 10:28:12 Jammin1 systemd[1]: Condition check resulted in Rebuild Hardware Database being skipped.
Dec 26 10:28:12 Jammin1 systemd[1]: Starting MD array monitor...
Dec 26 10:28:12 Jammin1 systemd[583]: mdmonitor.service: Executable /usr/lib/systemd/scripts/mdadm_env.sh missing, skipping: No such file or directory
Dec 26 10:28:12 Jammin1 systemd[1]: Started MD array monitor.
 
dmesg
 
[    2.063326] sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/466 GiB)
[    2.063345] sd 0:0:0:0: [sda] Write Protect is off
[    2.063350] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    2.063381] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    2.063406] sd 2:0:0:0: [sdb] 976773168 512-byte logical blocks: (500 GB/466 GiB)
[    2.063429] sd 2:0:0:0: [sdb] Write Protect is off
[    2.063434] sd 2:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[    2.063470] sd 2:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 
[    2.068056] sd 0:0:0:0: [sda] supports TCG Opal
[    2.068061] sd 0:0:0:0: [sda] Attached SCSI disk
[    2.068367] sd 2:0:0:0: [sdb] supports TCG Opal
[    2.068373] sd 2:0:0:0: [sdb] Attached SCSI disk
 
[    2.126755] md/raid1:md0: active with 2 out of 2 mirrors
[    2.127217] md/raid1:md4: active with 2 out of 2 mirrors
[    2.127781] md0: detected capacity change from 0 to 32212254720
[    2.128700] md4: detected capacity change from 0 to 12876513280
[    2.129642] md/raid1:md3: active with 2 out of 2 mirrors
[    2.130200] md/raid1:md1: active with 2 out of 2 mirrors
[    2.130736] md3: detected capacity change from 0 to 32195477504
[    2.131574] md1: detected capacity change from 0 to 5364514816
[    2.132462] md/raid1:md2: active with 2 out of 2 mirrors
[    2.133526] md2: detected capacity change from 0 to 21458059264

Only thing that really sticks out is:

Dec 26 10:28:12 Jammin1 systemd[583]: mdmonitor.service: Executable /usr/lib/systemd/scripts/mdadm_env.sh missing, skipping: No such file or directory
0 Likes

#2

Hmm, I don’t use raid at all on my end, but the issue sounds familiar. Look also here.

0 Likes

#3

Hmmm…seems that file was put in for SUSE, and just generates that log message. Like other posters there, my arrays are starting/working fine.

Probably going off on a tangent here, but what kind of entry in dmesg/journal can I look for to see when the system transfers from the initramfs? Because, since the raid arrays are assembled in the initramfs, I would think that the underlying devices /dev/sd[ab][2-6] would be locked…and nothing but mdadm should be writing to them; if those errors are thrown after the system goes ‘live’, then they are probably nothing more than just log ‘noise’.

0 Likes

#4

Seems to be fixed by v240.156-1.

0 Likes

#5

Thx for letting me know.

0 Likes

#6

I confirm it! :slight_smile:

0 Likes

#7

No more udev errors here either.

0 Likes

closed #8

This topic was automatically closed after 90 days. New replies are no longer allowed.

0 Likes