Logrotate fails

deemon · 4 September 2020 00:04

$ systemctl status logrotate.service 
● logrotate.service - Rotate log files
     Loaded: loaded (/usr/lib/systemd/system/logrotate.service; static)
     Active: failed (Result: exit-code) since Fri 2020-09-04 00:00:34 EEST; 2h 57min ago
TriggeredBy: ● logrotate.timer
       Docs: man:logrotate(8)
             man:logrotate.conf(5)
    Process: 62321 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf (code=exited, status=3)
   Main PID: 62321 (code=exited, status=3)

sept  04 00:00:34 Zen systemd[1]: Starting Rotate log files...
sept  04 00:00:34 Zen logrotate[62321]: error: state file /var/lib/logrotate.status is already locked
sept  04 00:00:34 Zen logrotate[62321]: logrotate does not support parallel execution on the same set of logfiles.
sept  04 00:00:34 Zen systemd[1]: logrotate.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
sept  04 00:00:34 Zen systemd[1]: logrotate.service: Failed with result 'exit-code'.
sept  04 00:00:34 Zen systemd[1]: Failed to start Rotate log files.

Googling gave this hint, but I am unsure what exactly I must do to apply it or if it’s even a right solution.

merlock · 4 September 2020 01:27

You can try:

logrotate --debug /etc/logrotate.conf

https://wiki.archlinux.org/index.php/Logrotate

deemon · 4 September 2020 14:39

Nothing seems to be out of the ordinary. When I just run it myself there is no errors.

$ logrotate --debug /etc/logrotate.conf

WARNING: logrotate in debug mode does nothing except printing debug messages!  Consider using verbose mode (-v) instead if this is not what you want.

reading config file /etc/logrotate.conf
including /etc/logrotate.d
reading config file cups
reading config file firewalld
reading config file libvirtd
reading config file libvirtd.lxc
reading config file libvirtd.qemu
reading config file lirc
reading config file samba
warning: 'monthly' overrides previously specified 'weekly'
Reading state from file: /var/lib/logrotate.status
Allocating hash table for state file, size 64 entries
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state

Handling 9 logs

rotating pattern: /var/log/cups/*_log  weekly (4 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/cups/access_log
  Now: 2020-09-04 17:36
  Last rotated at 2020-05-10 00:00
  log does not need rotating (log is empty)
considering log /var/log/cups/error_log
  Now: 2020-09-04 17:36
  Last rotated at 2020-05-09 00:00
  log does not need rotating (log is empty)

rotating pattern: /var/log/firewalld  weekly (4 rotations)
empty log files are rotated, only log files >= 1048576 bytes are rotated, old logs are removed
considering log /var/log/firewalld
  Now: 2020-09-04 17:36
  Last rotated at 2020-04-12 00:00
  log does not need rotating ('minsize' directive is used and the log size is smaller than the minsize value)

rotating pattern: /var/log/libvirt/libvirtd.log  weekly (4 rotations)
empty log files are rotated, only log files >= 102400 bytes are rotated, old logs are removed
considering log /var/log/libvirt/libvirtd.log
  log /var/log/libvirt/libvirtd.log does not exist -- skipping

rotating pattern: /var/log/libvirt/lxc/*.log  2097153 bytes (4 rotations)
empty log files are rotated, old logs are removed
considering log /var/log/libvirt/lxc/*.log
  log /var/log/libvirt/lxc/*.log does not exist -- skipping

rotating pattern: /var/log/libvirt/qemu/*.log  2097153 bytes (4 rotations)
empty log files are rotated, old logs are removed
considering log /var/log/libvirt/qemu/Recalbox-6.1.1-Dragonblaze.log
  Now: 2020-09-04 17:36
  Last rotated at 2020-06-04 00:00
  log does not need rotating (log size is below the 'size' threshold)
considering log /var/log/libvirt/qemu/Recalbox.log
  Now: 2020-09-04 17:36
  Last rotated at 2020-03-25 00:00
  log does not need rotating (log size is below the 'size' threshold)

rotating pattern: /var/log/lircd  weekly (4 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/lircd
  log /var/log/lircd does not exist -- skipping

rotating pattern: /var/log/samba/log.smbd /var/log/samba/log.nmbd /var/log/samba/*.log  weekly (4 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/samba/log.smbd
  log /var/log/samba/log.smbd does not exist -- skipping
considering log /var/log/samba/log.nmbd
  log /var/log/samba/log.nmbd does not exist -- skipping
considering log /var/log/samba/*.log
  log /var/log/samba/*.log does not exist -- skipping

rotating pattern: /var/log/wtmp  monthly (1 rotations)
empty log files are rotated, only log files >= 1048576 bytes are rotated, old logs are removed
considering log /var/log/wtmp
  Now: 2020-09-04 17:36
  Last rotated at 2020-05-05 00:00
  log does not need rotating ('minsize' directive is used and the log size is smaller than the minsize value)

rotating pattern: /var/log/btmp  monthly (1 rotations)
empty log files are rotated, old logs are removed
considering log /var/log/btmp
  Now: 2020-09-04 17:36
  Last rotated at 2020-09-01 00:00
  log does not need rotating (log has been rotated at 2020-09-01 00:00, which is less than a month ago)

$ sudo logrotate -v /etc/logrotate.conf

reading config file /etc/logrotate.conf
including /etc/logrotate.d
reading config file cups
reading config file firewalld
reading config file libvirtd
reading config file libvirtd.lxc
reading config file libvirtd.qemu
reading config file lirc
reading config file samba
warning: 'monthly' overrides previously specified 'weekly'
Reading state from file: /var/lib/logrotate.status
Allocating hash table for state file, size 64 entries
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state

Handling 9 logs

rotating pattern: /var/log/cups/*_log  weekly (4 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/cups/access_log
  Now: 2020-09-04 17:37
  Last rotated at 2020-05-10 00:00
  log does not need rotating (log is empty)
considering log /var/log/cups/error_log
  Now: 2020-09-04 17:37
  Last rotated at 2020-05-09 00:00
  log does not need rotating (log is empty)

rotating pattern: /var/log/firewalld  weekly (4 rotations)
empty log files are rotated, only log files >= 1048576 bytes are rotated, old logs are removed
considering log /var/log/firewalld
  Now: 2020-09-04 17:37
  Last rotated at 2020-04-12 00:00
  log does not need rotating ('minsize' directive is used and the log size is smaller than the minsize value)

rotating pattern: /var/log/libvirt/libvirtd.log  weekly (4 rotations)
empty log files are rotated, only log files >= 102400 bytes are rotated, old logs are removed
considering log /var/log/libvirt/libvirtd.log
  log /var/log/libvirt/libvirtd.log does not exist -- skipping

rotating pattern: /var/log/libvirt/lxc/*.log  2097153 bytes (4 rotations)
empty log files are rotated, old logs are removed
considering log /var/log/libvirt/lxc/*.log
  log /var/log/libvirt/lxc/*.log does not exist -- skipping

rotating pattern: /var/log/libvirt/qemu/*.log  2097153 bytes (4 rotations)
empty log files are rotated, old logs are removed
considering log /var/log/libvirt/qemu/Recalbox-6.1.1-Dragonblaze.log
  Now: 2020-09-04 17:37
  Last rotated at 2020-06-04 00:00
  log does not need rotating (log size is below the 'size' threshold)
considering log /var/log/libvirt/qemu/Recalbox.log
  Now: 2020-09-04 17:37
  Last rotated at 2020-03-25 00:00
  log does not need rotating (log size is below the 'size' threshold)

rotating pattern: /var/log/lircd  weekly (4 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/lircd
  log /var/log/lircd does not exist -- skipping

rotating pattern: /var/log/samba/log.smbd /var/log/samba/log.nmbd /var/log/samba/*.log  weekly (4 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/samba/log.smbd
  log /var/log/samba/log.smbd does not exist -- skipping
considering log /var/log/samba/log.nmbd
  log /var/log/samba/log.nmbd does not exist -- skipping
considering log /var/log/samba/*.log
  log /var/log/samba/*.log does not exist -- skipping

rotating pattern: /var/log/wtmp  monthly (1 rotations)
empty log files are rotated, only log files >= 1048576 bytes are rotated, old logs are removed
considering log /var/log/wtmp
  Now: 2020-09-04 17:37
  Last rotated at 2020-05-05 00:00
  log does not need rotating ('minsize' directive is used and the log size is smaller than the minsize value)

rotating pattern: /var/log/btmp  monthly (1 rotations)
empty log files are rotated, old logs are removed
considering log /var/log/btmp
  Now: 2020-09-04 17:37
  Last rotated at 2020-09-01 00:00
  log does not need rotating (log has been rotated at 2020-09-01 00:00, which is less than a month ago)

merlock · 4 September 2020 15:03

You could try renaming /var/lib/logrotate.status to /var/lib/logrotate.status.old, and running logrotate from the service file:

sudo systemctl start logrotate.service

Then check the status.

deemon · 4 September 2020 15:07

It seems to have worked without errors. We’ll see tomorrow if this continues to be errorless overnight also.

deemon · 4 September 2020 22:16

Bleh…

sept  05 00:00:24 Zen systemd[1]: Starting Rotate log files...
sept  05 00:00:24 Zen logrotate[99138]: error: state file /var/lib/logrotate.status is already locked
sept  05 00:00:24 Zen logrotate[99138]: logrotate does not support parallel execution on the same set of logf>
sept  05 00:00:24 Zen systemd[1]: logrotate.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
sept  05 00:00:24 Zen systemd[1]: logrotate.service: Failed with result 'exit-code'.
sept  05 00:00:24 Zen systemd[1]: Failed to start Rotate log files.

again.

anothermindbomb · 4 September 2020 22:23

Are you perhaps trying to logrotate symlinks or named pipes? I see the status=3/NOTIMPLEMENTED and whilst I’ve no experience with logrotate failures I’ve seen installers (I’m looking at you Informatica PowerCentre) fail due to named pipes sitting around in directories where they were not expected…

merlock · 5 September 2020 01:46

Run

sudo logrotate -v /etc/logrotate.conf

again.

Another thing you might try is to do a replacement/override for logrotate.service.

See: systemd - ArchWiki

ETA: add --skip-state-lock

deemon · 5 September 2020 10:41

what exactly I must do for that? the original is:

$ cat /usr/lib/systemd/system/logrotate.service 
[Unit]
Description=Rotate log files
Documentation=man:logrotate(8) man:logrotate.conf(5)
RequiresMountsFor=/var/log
ConditionACPower=true

[Service]
Type=oneshot
ExecStart=/usr/sbin/logrotate /etc/logrotate.conf

# performance options
Nice=19
IOSchedulingClass=best-effort
IOSchedulingPriority=7

# hardening options
#  details: https://www.freedesktop.org/software/systemd/man/systemd.exec.html
#  no ProtectHome for userdir logs
#  no PrivateNetwork for mail deliviery
#  no NoNewPrivileges for third party rotate scripts
LockPersonality=true
MemoryDenyWriteExecute=true
PrivateDevices=true
PrivateTmp=true
ProtectClock=true
ProtectControlGroups=true
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectSystem=full
RestrictNamespaces=true
RestrictRealtime=true

I assume from archwiki I must do sudo systemctl edit logrotate.service but this opens completely empty editor, where I should write then the overwrite things… but what and how?
Would this be OK:

ExecStart=
ExecStart=/usr/sbin/logrotate --skip-state-lock /etc/logrotate.conf

or do I need to add the category also, like this (currently trying this one out; will see at midnight if this works out. manual execution via systemctl start logrotate worked again, but … so did the previous try):

[Service]
ExecStart=
ExecStart=/usr/sbin/logrotate --skip-state-lock /etc/logrotate.conf

or should I actually make any of the protect or lock options in the end of the file there false instead of current true? LockPersonality maybe? NoNewPrivileges?

merlock · 5 September 2020 13:37

That’s what you use.

Start there, then try the other things, one at a time.

system · 8 September 2020 13:37

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.