Mosquitto fails to auto-start at boot

Hello, I installed mosquitto (version 2.0.8 which is the latest) and enabled auto start with sudo systemctl enable mosquitto.service.
However, it fails to start automatically after reboot with this error:

    systemctl status mosquitto
    ● mosquitto.service - Mosquitto MQTT Broker daemon
         Loaded: loaded (/usr/lib/systemd/system/mosquitto.service; enabled; vendor preset: disabled)
         Active: failed (Result: exit-code) since Fri 2021-03-26 12:01:14 CET; 48s ago
        Process: 638 ExecStart=/usr/bin/mosquitto -c /etc/mosquitto/mosquitto.conf (code=exited, status=1/FAILURE)
       Main PID: 638 (code=exited, status=1/FAILURE)

    mar 26 12:01:14 nas systemd[1]: mosquitto.service: Scheduled restart job, restart counter is at 5.
    mar 26 12:01:14 nas systemd[1]: Stopped Mosquitto MQTT Broker daemon.
    mar 26 12:01:14 nas systemd[1]: mosquitto.service: Start request repeated too quickly.
    mar 26 12:01:14 nas systemd[1]: mosquitto.service: Failed with result 'exit-code'.
    mar 26 12:01:14 nas systemd[1]: Failed to start Mosquitto MQTT Broker daemon.

It is not really helpful, is it?
If I start it manually with sudo systemctl start mosquitto.service it works.
Can you please help me to debug this problem? Could it be something permission-related?

Thanks,
Andrea

EDIT
I tried running the unit’s command and it appears to work correctly:
sudo -u mosquitto /usr/bin/mosquitto -c /etc/mosquitto/mosquitto.conf. It starts and I am able to see connections and publish messages.

Permissons look also good, at least to me:

    sudo ls -als /etc/mosquitto/
     4 drwxr-xr-x  2 mosquitto root  4096 23 mar 14.47 .
     4 drwxr-xr-x 74 root      root  4096 23 gen  2013 ..
     4 -rw-r--r--  1 mosquitto root   230  6 mar 15.55 aclfile.example
    40 -rw-r--r--  1 mosquitto root 39393 22 mar 10.01 mosquitto.conf
    40 -rw-r--r--  1 mosquitto root 39196  6 mar 15.55 mosquitto.conf.pacnew
     4 -rw-r--r--  1 mosquitto root    23  6 mar 15.55 pskfile.example
     4 -rw-r--r--  1 mosquitto root   355  6 mar 15.55 pwfile.example
     4 -rw-r--r--  1 mosquitto root   743 17 mar 20.16 secret

Can you publish the file /usr/lib/systemd/system/mosquitto.service ?

Édit “Could it be something permission-related?” It could but as it works manually I’m thinking about dependencies.

Please handle .pacnew files after updates. See here.

Also reboot your system and run the following command and paste its output ( without doing sudo systemctl start mosquitto.service).

mosquitto -v -c /etc/mosquitto/mosquitto.conf

[Unit]
Description=Mosquitto MQTT Broker daemon
Requires=network.target

[Service]
User=mosquitto
ExecStart=/usr/bin/mosquitto -c /etc/mosquitto/mosquitto.conf
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure

NoNewPrivileges=yes
ProtectSystem=full
ProtectHome=true
PrivateTmp=true
PrivateDevices=true

[Install]
WantedBy=multi-user.target

Done, thanks. However, files were pretty much identical.
mosquitto -v -c /etc/mosquitto/mosquitto.conf gives:

1616771087: mosquitto version 2.0.8 starting
1616771087: Config loaded from /etc/mosquitto/mosquitto.conf.
1616771087: Opening ipv4 listen socket on port 1883.
1616771087: mosquitto version 2.0.8 running
1616771091: New connection from 192.168.1.85:54253 on port 1883.
...

It is working if started manually (with or without systemd), so maybe @Dulbi is right and dependencies might cause this issue.

Sorry to bump… Any other idea?

Solved with a quick and dirty hack:

sudo systemctl edit mosquitto.service

# Add following lines to override default restart behaviour:
[Service]
Restart=always
RestartSec=15
StartLimitInterval=150
StartLimitBurst=8

So basically, restarting the service for a longer time seems to fix my issue.
This is not the best way to work it out, but it works. I honestly don’t have any more time to properly fix this.
It looks like a dependency problem after all, as @Dulbi suggested.
Thanks for your time anyway.

Seems like I had an issue a while back with a service not starting when it involved network.target (which I never found). I changed to NetworkManager.service and the service started working.

Try changing in the .service file:

Requires=network.target
To:
After=NetworkManager.service

Doesn’t network manager require a GUI? I am running headless, no desktop environment is installed

Then After=what-ever-network.service you have.

This looked prmising, but unfortunately didn’t work.
This is my custom service file:

[Unit]
After=systemd-networkd.service

Still, mosquitto did not start automatically after a reboot.
I’ll stick with my improper solution.
Again, thanks