Permissions of /dev/null reset to crw-rw-r-- after every shutdown, impedes boot

Support
1

Hi all, I’m new here. Hope this is the right place to troubleshoot.
So I was customizing my new install of Manjaro last week, and I noticed my gufw settings had not persisted after a reboot. So I flipped the switch back to on and rebooted to check if the firewall would automatically start like its supposed to. However, I was met with a blank screen and the only thing I could do was to log in via tty to troubleshoot. After bumbling around for far too long (the shell literally told me -bash: /dev/null: Permission denied, yet I still spent much time looking elsewhere), I discovered that /dev/null’s permissions were set incorrectly to crw-rw-r- -, keeping the window environment from starting. Setting the permissions to crw-rw-rw- allowed me to start the window environment with startx, although it hangs for a random amount of time before starting Cinnamon, which is rather irksome. Furthermore, shutting the computer resets the permissions, requiring me to run sudo chmod o+w /dev/null every time I start the computer.
I have been stumbling through this week manually setting the permissions every time I boot the computer, but I’d prefer to have a system that works the way its supposed to and actually boots quickly. Does anyone have any idea what went wrong here? My kernel is Linux 5.15.32-1, if it’s any use. Gufw did indeed turn itself off again, but I believe it has nothing to do with the issue as removing it does not solve the issue.
For what it’s worth:
Output of pacman -Qe: https://pastebin.com/Mh0q2QRN
Output of pacman -Q: https://pastebin.com/0reN053V
Please let me know if I can provide any additional information that would help solve the problem, or if I didn’t explain well enough. Thanks!

EDIT:

System information, whoops :) 
System:
  Kernel: 5.15.32-1-MANJARO arch: x86_64 bits: 64 compiler: gcc v: 11.2.0
    parameters: BOOT_IMAGE=/boot/vmlinuz-5.15-x86_64
    root=UUID=b3781741-cb52-434b-a88f-8ce976e35c48 rw quiet
    resume=UUID=311a8a64-76e7-4c00-948d-ceb76d7b484b udev.log_priority=3
  Desktop: Cinnamon v: 5.2.7 tk: GTK v: 3.24.33 vt: 2 dm: LightDM v: 1.30.0
    Distro: Manjaro Linux base: Arch Linux
Machine:
  Type: Laptop System: LENOVO product: 2342CTO v: ThinkPad T430
    serial: <superuser required> Chassis: type: 10 serial: <superuser required>
  Mobo: LENOVO model: 2342CTO serial: <superuser required>
    UEFI-[Legacy]: LENOVO v: G1ETC2WW (2.82 ) date: 08/07/2019
Battery:
  ID-1: BAT0 charge: 50.8 Wh (87.1%) condition: 58.3/93.6 Wh (62.3%)
    volts: 12.1 min: 11.1 model: LGC 45N1011 type: Li-ion serial: <filter>
    status: discharging
Memory:
  RAM: total: 7.47 GiB used: 2.05 GiB (27.5%)
  RAM Report:
    permissions: Unable to run dmidecode. Root privileges required.
CPU:
  Info: model: Intel Core i5-3210M bits: 64 type: MT MCP arch: Ivy Bridge
    family: 6 model-id: 0x3A (58) stepping: 9 microcode: 0x21
  Topology: cpus: 1x cores: 2 tpc: 2 threads: 4 smt: enabled cache:
    L1: 128 KiB desc: d-2x32 KiB; i-2x32 KiB L2: 512 KiB desc: 2x256 KiB
    L3: 3 MiB desc: 1x3 MiB
  Speed (MHz): avg: 1227 high: 1293 min/max: 1200/3100 scaling:
    driver: intel_cpufreq governor: schedutil cores: 1: 1197 2: 1293 3: 1214
    4: 1207 bogomips: 19960
  Flags: acpi aes aperfmperf apic arat arch_perfmon avx bts clflush cmov
    constant_tsc cpuid cpuid_fault cx16 cx8 de ds_cpl dtes64 dtherm dts epb ept
    erms est f16c flexpriority flush_l1d fpu fsgsbase fxsr ht ibpb ibrs ida
    lahf_lm lm mca mce md_clear mmx monitor msr mtrr nonstop_tsc nopl nx pae
    pat pbe pcid pclmulqdq pdcm pebs pge pln pni popcnt pse pse36 pti pts
    rdrand rdtscp rep_good sep smep ss ssbd sse sse2 sse4_1 sse4_2 ssse3 stibp
    syscall tm tm2 tpr_shadow tsc tsc_deadline_timer vme vmx vnmi vpid x2apic
    xsave xsaveopt xtopology xtpr
  Vulnerabilities:
  Type: itlb_multihit status: KVM: VMX disabled
  Type: l1tf
    mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable
  Type: mds mitigation: Clear CPU buffers; SMT vulnerable
  Type: meltdown mitigation: PTI
  Type: spec_store_bypass
    mitigation: Speculative Store Bypass disabled via prctl and seccomp
  Type: spectre_v1
    mitigation: usercopy/swapgs barriers and __user pointer sanitization
  Type: spectre_v2 mitigation: Retpolines, IBPB: conditional, IBRS_FW,
    STIBP: conditional, RSB filling
  Type: srbds status: Vulnerable: No microcode
  Type: tsx_async_abort status: Not affected
Graphics:
  Device-1: Intel 3rd Gen Core processor Graphics vendor: Lenovo driver: i915
    v: kernel ports: active: LVDS-1
    empty: DP-1, DP-2, DP-3, HDMI-A-1, HDMI-A-2, HDMI-A-3, VGA-1
    bus-ID: 00:02.0 chip-ID: 8086:0166 class-ID: 0300
  Device-2: Chicony thinkpad t430s camera type: USB driver: uvcvideo
    bus-ID: 1-1.6:4 chip-ID: 04f2:b2da class-ID: 0e02
  Display: x11 server: X.Org v: 1.21.1.3 driver: X: loaded: modesetting
    unloaded: fbdev,vesa gpu: i915 display-ID: :1 screens: 1
  Screen-1: 0 s-res: 1600x900 s-dpi: 96 s-size: 423x238mm (16.65x9.37")
    s-diag: 485mm (19.11")
  Monitor-1: LVDS-1 model: AU Optronics 0x213e built: 2010 res: 1600x900
    hz: 60 dpi: 132 gamma: 1.2 size: 309x174mm (12.17x6.85") diag: 355mm (14")
    ratio: 16:9 modes: 1600x900
  Message: Unable to show GL data. Required tool glxinfo missing.
Audio:
  Device-1: Intel 7 Series/C216 Family High Definition Audio vendor: Lenovo
    driver: snd_hda_intel v: kernel bus-ID: 00:1b.0 chip-ID: 8086:1e20
    class-ID: 0403
  Sound Server-1: ALSA v: k5.15.32-1-MANJARO running: yes
  Sound Server-2: JACK v: 1.9.20 running: no
  Sound Server-3: PulseAudio v: 15.0 running: yes
  Sound Server-4: PipeWire v: 0.3.49 running: yes
Network:
  Device-1: Intel 82579LM Gigabit Network vendor: Lenovo driver: e1000e
    v: kernel port: 6080 bus-ID: 00:19.0 chip-ID: 8086:1502 class-ID: 0200
  IF: enp0s25 state: down mac: <filter>
  Device-2: Intel Centrino Advanced-N 6205 [Taylor Peak] driver: iwlwifi
    v: kernel pcie: gen: 1 speed: 2.5 GT/s lanes: 1 bus-ID: 03:00.0
    chip-ID: 8086:0085 class-ID: 0280
  IF: wlp3s0 state: up mac: <filter>
  IP v4: <filter> type: dynamic noprefixroute scope: global
    broadcast: <filter>
  IP v6: <filter> type: noprefixroute scope: link
  WAN IP: <filter>
Bluetooth:
  Message: No bluetooth data found.
Logical:
  Message: No logical block device data found.
RAID:
  Message: No RAID data found.
Drives:
  Local Storage: total: 238.47 GiB used: 11.93 GiB (5.0%)
  ID-1: /dev/sda maj-min: 8:0 vendor: Crucial model: CT256M550SSD1
    size: 238.47 GiB block-size: physical: 4096 B logical: 512 B speed: 6.0 Gb/s
    type: SSD serial: <filter> rev: MU01 scheme: MBR
  SMART Message: Unknown smartctl error. Unable to generate data.
  Optical-1: /dev/sr0 vendor: PLDS model: DVD-RW DS8A8SH rev: KU54
    dev-links: cdrom
  Features: speed: 24 multisession: yes audio: yes dvd: yes
    rw: cd-r,cd-rw,dvd-r,dvd-ram state: running
Partition:
  ID-1: / raw-size: 229.67 GiB size: 225 GiB (97.97%) used: 11.93 GiB (5.3%)
    fs: ext4 block-size: 4096 B dev: /dev/sda1 maj-min: 8:1 label: N/A
    uuid: b3781741-cb52-434b-a88f-8ce976e35c48
Swap:
  Kernel: swappiness: 60 (default) cache-pressure: 100 (default)
  ID-1: swap-1 type: partition size: 8.8 GiB used: 0 KiB (0.0%) priority: -2
    dev: /dev/sda2 maj-min: 8:2 label: swap
    uuid: 311a8a64-76e7-4c00-948d-ceb76d7b484b
Unmounted:
  Message: No unmounted partitions found.
USB:
  Hub-1: 1-0:1 info: Full speed or root hub ports: 3 rev: 2.0 speed: 480 Mb/s
    chip-ID: 1d6b:0002 class-ID: 0900
  Hub-2: 1-1:2 info: Intel Integrated Rate Matching Hub ports: 6 rev: 2.0
    speed: 480 Mb/s chip-ID: 8087:0024 class-ID: 0900
  Device-1: 1-1.6:4 info: Chicony thinkpad t430s camera type: Video
    driver: uvcvideo interfaces: 2 rev: 2.0 speed: 480 Mb/s power: 200mA
    chip-ID: 04f2:b2da class-ID: 0e02
  Hub-3: 2-0:1 info: Full speed or root hub ports: 3 rev: 2.0
    speed: 480 Mb/s chip-ID: 1d6b:0002 class-ID: 0900
  Hub-4: 2-1:2 info: Intel Integrated Rate Matching Hub ports: 8 rev: 2.0
    speed: 480 Mb/s chip-ID: 8087:0024 class-ID: 0900
  Hub-5: 3-0:1 info: Hi-speed hub with single TT ports: 4 rev: 2.0
    speed: 480 Mb/s chip-ID: 1d6b:0002 class-ID: 0900
  Hub-6: 4-0:1 info: Super-speed hub ports: 4 rev: 3.0 speed: 5 Gb/s
    chip-ID: 1d6b:0003 class-ID: 0900
Sensors:
  System Temperatures: cpu: 40.0 C mobo: N/A
  Fan Speeds (RPM): fan-1: 2562
Info:
  Processes: 225 Uptime: 34m wakeups: 10 Init: systemd v: 250 tool: systemctl
  Compilers: gcc: 11.2.0 clang: 13.0.1 Packages: pacman: 1160 lib: 360
  flatpak: 0 Shell: Bash v: 5.1.16 running-in: gnome-terminal inxi: 3.3.15
2

Did you overwrite the file?

LANG=C file /dev/null

LANG=C stat /dev/null

If so, then you need to recreate it:

rm -f /dev/null; mknod -m 666 /dev/null c 1 3
1 Like
3

The /dev tree is a virtual tree created by the kernel at boot time. It holds a virtual filesystem (devfs) providing access to various devices.

$ stat /dev
  File: /dev
  Size: 4360            Blocks: 0          IO Block: 4096   directory
Device: 0,5     Inode: 1           Links: 21
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2022-05-07 07:14:27.464101087 +0200
Modify: 2022-05-07 09:09:34.556484932 +0200
Change: 2022-05-07 09:09:34.556484932 +0200
 Birth: -
$ stat /dev/null
  File: /dev/null
  Size: 0               Blocks: 0          IO Block: 4096   character special file
Device: 0,5     Inode: 4           Links: 1     Device type: 1,3
Access: (0666/crw-rw-rw-)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2022-05-07 07:14:27.784101090 +0200
Modify: 2022-05-07 07:14:27.784101090 +0200
Change: 2022-05-07 07:14:27.784101090 +0200
 Birth: -

My guess is is you may have compiled your own kernel - fiddling with flags - perhaps you have a kernel argument modifying the behavior - perpahs a sysctl.d configuration - the possibilities are next to endless.

Suffice to say - the behavior is not default - it is a result of fiddling with system properties.

To find what is causing your issue you will have to backtrace your manual changes - validate the scripts you have run …

1 Like
4

@megavolt LANG=C file /dev/null returns

/dev/null: character special (1/3)

so I think the file is okay. I recreated it as you suggested just to check, but the problem still persists.

@linux-aarhus I have not messed with the kernel or compiled my own, unless it can be done on accident. :slight_smile: I don’t know enough about that to even know why I would bother. Is it possible some other program modified the kernel without my knowledge? How can I check and see if my kernel is abnormal? Ought I to change my kernel to see if that fixes anything?
Thanks for your replies.

5

Welcome to Manjaro! :smiling_face_with_three_hearts:

  1. Please read the information behind this link. It will help you to post necessary information. [HowTo] Provide System Information
  2. Please press the three dots … below your post and then press the :pencil2:
  • If you give us information about your system, we can see what we’re talking about and make better suggestions.
  • You can do this by using inxi in a terminal or in console.
inxi --admin --verbosity=7 --filter --no-host --width
  • Personally identifiable information such as serial numbers and MAC addresses are filtered out by this command
  • Presenting the information in this way allows everyone to be familiar with the format and quickly find the items they need without missing anything.
  1. Copy the output from inxi (including the command) and paste it into your post.
  • To make it more readable, add 3 backticks ``` on an extra line before and after the pasted text.
1 Like
6

In that process you have created - inadvertently - the issue - no one can help you - unless you roll back the customization you made…

Not directly - as I said - the possibilities are endless

7

Right, I understand that. The customizations I made to the system were pretty surface level (at least I thought so), not to mention I don’t know where to start. I’ll try undoing what I remember doing. I will update with news later. Thanks

EDIT: Is it possible to monitor /dev/null and log when its permissions are being changed? Can the ‘audit’ tool be setup to log such a thing?

EDIT2: Well, color me stumped. I reverted everything I did to the system that day in terms of packages installed and system files modified. I tried installing version 5.17.1-3 of the kernel, all to no avail. I can’t think of anything else I may have changed. Unless there is a way to monitor the file, I don’t know how to figure out what is changing the permissions.

8

Okay, figured it out.
Slipped through the cracks because I was editing files days before anything became a problem, but this guy’s solution pointed me to my udev rules. I had created a custom rules file to try and get my fingerprint reader working. I never figured out how to get it to work, but I forgot to remove the rules file afterwards. No idea why the file was messing things up the way it did, if anyone can explain why I’d be interested to know.
So for anyone else with the same problem- see if your udev rules are funky. Thank you all for your help!

9
10

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.