Borg/Borgmatic error

djaywhite · 17 October 2020 06:20

Borg/Borgmatic error

I’m trying to use Borgmatic (because it also backs up Databases).

It’s a sort of wrapper around Borg.

It uses systemd to do the scheduling.

When I run it manually using :

$ sudo borgmatic --verbosity 1 --files

It WORKS. It does the backup and the pruning perfectly.

However…

In the systemd borgmatic.service file the command to kick off the backup is:

ExecStart=systemd-inhibit
    --who="borgmatic"
    --why="Prevent interrupting scheduled backup"
    /usr/bin/borgmatic --syslog-verbosity 1

This FAILS, giving me the following:

INFO /data/backup/borg_a: Pruning archives
INFO Failed to create/acquire the lock /data/backup/borg_a/lock.exclusive
([Errno 13] Permission denied: '/data/backup/borg_a/lock.exclusive')

Ahh, thinks me, it’s because it’s needs to be run as sudo.

So in the borgmatic.service file I use:

/usr/bin/sudo borgmatic --syslog-verbosity 1

instead of

/usr/bin/borgmatic --syslog-verbosity 1

And then I get :

sudo: PERM_SUDOERS: setresuid(-1, 1, -1): Operation not permitted
sudo: no valid sudoers sources found, quitting
sudo: error initializing audit plugin sudoers_audit
/usr/bin/sudo failed with exit status 1.
borgmatic.service: Main process exited, code=exited, status=1/FAILURE
borgmatic.service: Failed with result 'exit-code'.
Failed to start borgmatic backup.

It seems obvious that it NEEDS the sudo password.

But how do I supply it automatically ???

megavolt · 17 October 2020 06:26

I would say, add it to /etc/sudoers:

<myusername> ALL = (root) NOPASSWD: /usr/bin/borgmatic

djaywhite · 17 October 2020 06:29

@megavolt Thanks for the quick response, I have never come across this before.
I shall try it in a few moments after making a .

I may be back shortly after I’ve screwed something else up , lol.

xabbu · 17 October 2020 07:40

Please do not use sudo in systemd files. All systemd system service units are running as root. Only if the systemd unit files contains a line User= it would run the commands as this user. Otherwise it is always root.

djaywhite · 17 October 2020 07:43

@xabbu. Thanks but I’m confused, if I don’t use sudo, which I have tried, then it just gives me Permission Denied
So how do I get it to run ?

xabbu · 17 October 2020 07:59

It is indeed a little bit wired.

What happens if you lock in as root, or use su - (the minus sign is important) and run the borgmatic command? Also do your mount point need some special environment variables? The environment of systemd services is different that the environment of your normal user, even with sudo.

djaywhite · 17 October 2020 08:08

@xabbu,

I can manually run a backup using:
sudo borgmatic --verbosity 1 --files
It works fine.
The mount is a usb drive which is mounted via systemd anyhow.
I don’t have any special environment variables at all.

xabbu · 17 October 2020 08:11

But it is with sudo. That is different!

Use su - so you will get a similar but not the same environment. And of course do not use sudo after you locked in as root.

djaywhite · 17 October 2020 08:14

I did su - got a new red colored prompt and ran borgmatic --verbosity 1 --files
aka without the sudo in front and everything ran fine, no problems.

djaywhite · 17 October 2020 08:16

BTW. How do I now leave the su - prompt ?

xabbu · 17 October 2020 08:17

exit

or ctrl+d

djaywhite · 17 October 2020 08:17

thx

xabbu · 17 October 2020 08:19

can you post

systemctl status borgmatic.service

and

systemctl cat borgmatic.service

btw., which filesystem is on this usb drive?

djaywhite · 17 October 2020 08:22

● borgmatic.service - borgmatic backup
     Loaded: loaded (/etc/systemd/system/borgmatic.service; static)
     Active: activating (start-pre) since Sat 2020-10-17 09:19:24 BST; 55s ago
TriggeredBy: ● borgmatic.timer
Cntrl PID: 22866 (sleep)
      Tasks: 1 (limit: 19086)
     Memory: 320.0K
     CGroup: /system.slice/borgmatic.service
             └─22866 /usr/bin/sleep 1m

Oct 17 09:19:24 david-j-white systemd[1]: Starting borgmatic backup...

# /etc/systemd/system/borgmatic.service
[Unit]
Description=borgmatic backup
Wants=network-online.target
After=network-online.target
ConditionACPower=true

[Service]
Type=oneshot

# Security settings for systemd running as root
# For more details about this settings check the systemd manuals
# https://www.freedesktop.org/software/systemd/man/systemd.exec.html
LockPersonality=true
# Certain borgmatic features like Healthchecks integration need MemoryDenyWriteExecute to be off.
# But you can try setting it to "yes" for improved security if you don't use those features.
MemoryDenyWriteExecute=no
NoNewPrivileges=yes
PrivateDevices=yes
PrivateTmp=yes
ProtectClock=yes
ProtectControlGroups=yes
ProtectHostname=yes
ProtectKernelLogs=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 AF_NETLINK
RestrictNamespaces=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
SystemCallArchitectures=native
SystemCallFilter=@system-service
# Restrict write access
# Change to 'ProtectSystem=strict' and uncomment 'ProtectHome' to make the whole file
# system read-only be default and uncomment 'ReadWritePaths' for the required write access.
# Add local repositroy paths to the list of 'ReadWritePaths' like '-/mnt/my_backup_drive'.
ProtectSystem=full
# ProtectHome=read-only
ReadWritePaths=-/root/.config/borg -/root/.cache/borg -/root/.borgmatic -/data/backup/borg_a

CapabilityBoundingSet=CAP_DAC_READ_SEARCH CAP_NET_RAW

# Lower CPU and I/O priority.
Nice=19
CPUSchedulingPolicy=batch
IOSchedulingClass=best-effort
IOSchedulingPriority=7
IOWeight=100

Restart=no
# Prevent rate limiting of borgmatic log events. If you are using an older version of systemd that
# doesn't support this (pre-240 or so), you may have to remove this option.
LogRateLimitIntervalSec=0

# Delay start to prevent backups running during boot. Note that systemd-inhibit requires dbus and
# dbus-user-session to be installed.
ExecStartPre=sleep 1m
; ExecStart=systemd-inhibit --who="borgmatic" --why="Prevent interrupting scheduled backup" /root/.local/bin/borgmatic --syslog-verbosity 1
ExecStart=systemd-inhibit --who="borgmatic" --why="Prevent interrupting scheduled backup" /usr/bin/borgmatic --syslog-verbosity 1

xabbu · 17 October 2020 08:31

You use your own version of a borgmatic.service file. It has heavy security restrictions. And this is probably why it doesn’t work.

Please remove/rename this file and test the service file form the original borgmatic package in the repository.

If you remove your own service file and reload systemd

systemctl daemon-reload

it will automatically picked up.

check it with

systemctl status borgmatic.service

it should print

Loaded: loaded (/usr/lib/systemd/system/borgmatic.service; static)

If that works you can try and play with the security options until you make your own systemd file usable.

djaywhite · 17 October 2020 08:38

@xabbu Thanks. I’ll go away and try that.

djaywhite · 17 October 2020 09:32

@xabbu
Thanks for all your help thus far.

Without this forum and all you folks that help everyone, I would probably have chucked the towel in a good while ago.

Anyhoo…

I uninstalled borgmatic and then re-installed it.
I ran in manually first and it ran perfectly.

I looked (in pamac GUI files section) and found that the unit files (.service and .timer) are installed to:
/usr/lib/systemd/system/
instead of:
/etc/systemd/system/

I tried to enable borgmatic anyway and got the following:

$ systemctl enable borgmatic

The unit files have no installation config (WantedBy=, RequiredBy=, Also=,
Alias= settings in the [Install] section, and DefaultInstance= for template
units). This means they are not meant to be enabled using systemctl.
 
Possible reasons for having this kind of units are:
• A unit may be statically enabled by being symlinked from another unit's
  .wants/ or .requires/ directory.
• A unit's purpose may be to act as a helper for some other unit which has
  a requirement dependency on it.
• A unit may be started when needed via activation (socket, path, timer,
  D-Bus, udev, scripted systemctl call, ...).
• In case of template units, the unit is meant to be enabled with some
  instance name specified.

borgmatic.service (from /usr/lib/systemd/system/).

[Unit]
Description=borgmatic backup
Wants=network-online.target
After=network-online.target
ConditionACPower=true

[Service]
Type=oneshot

# Lower CPU and I/O priority.
Nice=19
CPUSchedulingPolicy=batch
IOSchedulingClass=best-effort
IOSchedulingPriority=7
IOWeight=100

Restart=no
# Prevent rate limiting of borgmatic log events. If you are using an older version of systemd that
# doesn't support this (pre-240 or so), you may have to remove this option.
LogRateLimitIntervalSec=0

# Delay start to prevent backups running during boot. Note that systemd-inhibit requires dbus and
# dbus-user-session to be installed.
ExecStartPre=sleep 1m
ExecStart=systemd-inhibit --who="borgmatic" --why="Prevent interrupting scheduled backup" /usr/bin/borgmatic --syslog-verbosity 1

borgmatic.timer (from /usr/lib/systemd/system/).

[Unit]
Description=Run borgmatic backup

[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target

So I’m guessing it knows where the unit files are at least.
Where do I go from here ?

xabbu · 17 October 2020 09:55

There was no need to do a reinstall. Before you do anything, make sure you removed or renamed your
borgmatic.service file form /etc/systemd/system/
This is very important! Do this first! A reinstall will not remove files you created yourself.

System packages install systemd files in /usr/lib/systemd/... This is the default location. Do not edit files in /usr/lib/systemd/..., they get overridden with the next update.

The /etc/systemd/system/ folder is for user. They can create own service files in this folder. Service files in this folder take precedence over files in /usr/lib/systemd/...
This means, if you create a file called borgmatic.service in /etc/systemd/system/ the service file in /usr/lib/systemd/system/ will be ignored.

You can only enable a service file that has a [Install] section. The borgmatic.service has no [Install] section and it doesn’t need one. The service is only started manually or with a timer. The timer that starts a service file has the same name but ends in .timer.

You should enable the timer of borgmatic but, you just ran enable borgmatic . By default systemd adds .service, if no suffix is used. This won’t work in your case.

You should always use a suffix to be precise.

systemctl status borgmatic.service

and not

systemctl status borgmatic

but in this case is the effect the same.

Enable the timer with

systemctl enable --now borgmatic.timer

The --now options starts the timer, otherwise you would need to start the timer or reboot.
Check with

systemctl list-timers

You also might want to start the service manually to check if it works.

djaywhite · 17 October 2020 11:12

@xabbu Thank you sir. It all works like you said above .
I made a copy of the timer and put it in /etc/systemd/system/ and set it to backup every minute,
overkill but just for checking and it works fine. Now set it to every 6hrs which is what I wanted.

I would have just gone with just borg + cron but borgmatic gives me the option to autobackup my databases as well. But not today, perhaps tommorrow.

Thanks for all the help. It is very much appreciated.

p.s. Resistance is NOT futile. (In this case at least)

system · 1 November 2020 11:12

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.