Which iscsi service to connect to iscsi at boot?

hello,

i have a iscsi LUN.on a Synology NAS.
I can connect to it just fine at boot on my Mint installation.

But it doesnt work in Manjaro.

Manually logging in also works fine.

I suspect it has something to with the iscsi service not running at boot.

iscsid.service is running, but iscsi.service and iscsi not once the system has booted:

services:

[manjaro-laptop me]# systemctl status iscsi
○ iscsi.service - Login and scanning of iSCSI devices
     Loaded: loaded (/usr/lib/systemd/system/iscsi.service; disabled; preset: disabled)
     Active: inactive (dead)
       Docs: man:iscsiadm(8)
             man:iscsid(8)
[manjaro-laptop me]# systemctl start iscsi.service
Job for iscsi.service failed because the control process exited with error code.
See "systemctl status iscsi.service" and "journalctl -xeu iscsi.service" for details.
[manjaro-laptop me]# systemctl status iscsi.service
× iscsi.service - Login and scanning of iSCSI devices
     Loaded: loaded (/usr/lib/systemd/system/iscsi.service; disabled; preset: disabled)
     Active: failed (Result: exit-code) since Mon 2024-03-25 23:03:59 CET; 13s ago
       Docs: man:iscsiadm(8)
             man:iscsid(8)
    Process: 1817 ExecStart=/usr/bin//iscsiadm -m node --loginall=automatic -W (code=exited, status=19)
   Main PID: 1817 (code=exited, status=19)
        CPU: 5ms

Mär 25 23:03:59 manjaro-laptop systemd[1]: Starting Login and scanning of iSCSI devices...
Mär 25 23:03:59 manjaro-laptop iscsiadm[1817]: iscsiadm: Could not login all leading-login portals
Mär 25 23:03:59 manjaro-laptop systemd[1]: iscsi.service: Main process exited, code=exited, status=19/n/a
Mär 25 23:03:59 manjaro-laptop systemd[1]: iscsi.service: Failed with result 'exit-code'.
Mär 25 23:03:59 manjaro-laptop systemd[1]: Failed to start Login and scanning of iSCSI devices.
[manjaro-laptop me]# systemctl status iscsi
iscsid.service      iscsid.socket       iscsi-init.service  iscsi.service       iscsiuio.service    iscsiuio.socket     
[manjaro-laptop me]# systemctl status iscsi
iscsid.service      iscsid.socket       iscsi-init.service  iscsi.service       iscsiuio.service    iscsiuio.socket     
[manjaro-laptop me]# systemctl status iscsid.service 
● iscsid.service - Open-iSCSI
     Loaded: loaded (/usr/lib/systemd/system/iscsid.service; enabled; preset: disabled)
     Active: active (running) since Mon 2024-03-25 23:01:42 CET; 3min 7s ago
TriggeredBy: ● iscsid.socket
       Docs: man:iscsid(8)
             man:iscsiuio(8)
             man:iscsiadm(8)
   Main PID: 937 (iscsid)
     Status: "Ready to process requests"
      Tasks: 1 (limit: 18835)
     Memory: 5.2M (peak: 5.4M)
        CPU: 9ms
     CGroup: /system.slice/iscsid.service
             └─937 /usr/bin//iscsid -f

Mär 25 23:01:42 manjaro-laptop systemd[1]: Starting Open-iSCSI...
Mär 25 23:01:42 manjaro-laptop systemd[1]: Started Open-iSCSI.
[manjaro-laptop me]# iscsiadm --mode node --targetname=iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c --portal 192.168.1.101:3260 --login
Logging in to [iface: default, target: iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c, portal: 192.168.1.101,3260]
Login to [iface: default, target: iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c, portal: 192.168.1.101,3260] successful.
[manjaro-laptop me]# mount /dev/sda1 /mnt/iscsi-vmware/
[manjaro-laptop me]#

I guess i need to enable the service, but its not possible:

[manjaro-laptop me]# journalctl -u iscsi.service | tail -100 
Mär 25 23:03:59 manjaro-laptop systemd[1]: Starting Login and scanning of iSCSI devices...
Mär 25 23:03:59 manjaro-laptop iscsiadm[1817]: iscsiadm: Could not login all leading-login portals
Mär 25 23:03:59 manjaro-laptop systemd[1]: iscsi.service: Main process exited, code=exited, status=19/n/a
Mär 25 23:03:59 manjaro-laptop systemd[1]: iscsi.service: Failed with result 'exit-code'.
Mär 25 23:03:59 manjaro-laptop systemd[1]: Failed to start Login and scanning of iSCSI devices.
[23:46]
[manjaro-laptop me]# systemctl start iscsi.service
Job for iscsi.service failed because the control process exited with error code.
See "systemctl status iscsi.service" and "journalctl -xeu iscsi.service" for details.
[manjaro-laptop me]# systemctl status iscsi.service
× iscsi.service - Login and scanning of iSCSI devices
     Loaded: loaded (/usr/lib/systemd/system/iscsi.service; disabled; preset: disabled)
     Active: failed (Result: exit-code) since Mon 2024-03-25 23:03:59 CET; 13s ago
       Docs: man:iscsiadm(8)
             man:iscsid(8)
    Process: 1817 ExecStart=/usr/bin//iscsiadm -m node --loginall=automatic -W (code=exited, status=19)
   Main PID: 1817 (code=exited, status=19)
        CPU: 5ms

Mär 25 23:03:59 manjaro-laptop systemd[1]: Starting Login and scanning of iSCSI devices...
Mär 25 23:03:59 manjaro-laptop iscsiadm[1817]: iscsiadm: Could not login all leading-login portals
Mär 25 23:03:59 manjaro-laptop systemd[1]: iscsi.service: Main process exited, code=exited, status=19/n/a
Mär 25 23:03:59 manjaro-laptop systemd[1]: iscsi.service: Failed with result 'exit-code'.
Mär 25 23:03:59 manjaro-laptop systemd[1]: Failed to start Login and scanning of iSCSI devices.

Also my boot and shutdown times take like 3 minutes each since some days.
Im not sure if this is related to the iscsi issue.

So not booting from that LUN?

It’s been a long time since I’ve used iSCSI. You have the proper modules loaded? For not booting, it used to be as simple as: loading a module, starting the service, and discover/add.

The Arch iSCSI wiki (for booting which is the only one over there) seems to use:

iscsi_tcp iscsi_ibft libiscsi libiscsi_tcp scsi_transport_iscsi

How did Mint connect to it via “just fine”? You still had to use isciadm on the command line?

No, im not booting from the LUN, just connecting to it, so I can use it as a network drives more or less (SMB is not an option in this case).

On Mint I dont have to do anything manually.
It is mounted at boot.
No running iscsiadm manually.

I dont know if I have the proper modules loaded.
How do I check that?

But if I can just connect to it once Manjaro has booted, without manually loading any modules shouldnt they be loaded?

For Mint I just edited the /etc/iscsi/iscsid.conf and added the LUN UUID to fstab to mount it and it works every time when booting.

In Mint I used this in the cfg:

# Default for Debian and Ubuntu. Uncomment to activate.
iscsid.startup = /bin/systemctl start iscsid.socket

But im not sure, if thats the correct service for Arch/Manjaro?

Running
/bin/systemctl start iscsid.socket

manually gives me no errors, so it should be the correct one?

There are some errors logged related to iscsi before the boot splash screen, but they are visible too short to read.

In which logs can I possibly see those errors?

lsmod | grep iscsi
But I just installed open-iscsi myself, and the modules get loaded automatically when I use it.

systemctl start iscid to start
systemctl enable iscid to start on boot

journalctl -u iscsid (-f)

But if you leave the iscsid service stopped, you can run it manually with extra debug info with:
sudo iscsid -d 8 -c /etc/iscsi/iscsid.conf -i /etc/iscsi/initiatorname.iscsi -f

I just enabled a LUN on my DS923+, and with installing open-iscsi, enabling the service and using the commands:

sudo iscsiadm -m node --target IQN --portal NAS:3260
sudo iscsiadm -m node --target IQN --login

And I see the disk just fine. (Not using CHAP for simplicity.)

NAS being the hostname or IP, and IQN from the Synology DSM interface, or I can get it through: sudo iscsiadm -m discovery --portal NAS:3260 --type sendtargets

I was looking into getting this into iscsid.conf, which I think how this is done automatically. But in case I can’t get back to this today, I’ll leave you with that. Or possibly someone else that has used iSCSI this decade. (Not because it’s bad, just used to use it for work, now I don’t. :smirk: )

Sorry, I did not have time last week, but heres what I have now tested:

Yes, I have the same behavior, that iscsi modules get automatically loaded, when I use it.

The issue is not that they are not loaded, when manually connecting to iscsi, but when setting to do it at boot.

After booting only this module is loaded:

lsmod | grep iscsi
scsi_transport_iscsi   192512  1

When manually connecting to it with following command afterwards these modules are loaded:

sudo iscsiadm -m node --target iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c --portal 192.168.1.101:3260 --login

sudo mount -a

lsmod | grep iscsi
iscsi_tcp              28672  2
libiscsi_tcp           40960  1 iscsi_tcp
libiscsi               94208  2 libiscsi_tcp,iscsi_tcp
scsi_transport_iscsi   192512  4 libiscsi_tcp,iscsi_tcp,libiscsi

ll /mnt/iscsi-vmware/
i see my files here.

This doesnt work, even after the iscsi modules are loaded:

systemctl start iscid
Failed to start iscid.service: Unit iscid.service not found.

systemctl enable iscid
Failed to enable unit: Unit file iscid.service does not exist.

Well you tested, what is already working for me: Manually connecting to iscsi.

Can you reboot your PC and test, if it is being mounted at boot?

journalctl -u iscsid logs this. Im not sure what this means though, maybe a permissions issue?
Or is the issue, that is trying to connect via 10.8.0.1, which it shouldnt do?

Apr 03 04:38:56 manjaro-laptop systemd[1]: Starting Open-iSCSI...
Apr 03 04:38:56 manjaro-laptop systemd[1]: Started Open-iSCSI.
Apr 03 04:54:21 manjaro-laptop iscsid[967]: iscsid: Could not set session1 priority. READ/WRITE throughout and latency could be affected.
Apr 03 04:54:21 manjaro-laptop iscsid[967]: iscsid: Connection1:0 to [target: iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c, portal: 192.168.1.101,3260] through [iface: default] is operational now
Apr 03 04:56:22 manjaro-laptop iscsid[967]: iscsid: Connection-1:0 to [target: iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c, portal: 10.8.0.1,3260] through [iface: default] is shutdown.
Apr 03 04:56:22 manjaro-laptop iscsid[967]: iscsid: connection-1:0 IPC qtask write failed: Broken pipe
Apr 03 05:00:53 manjaro-laptop iscsid[967]: iscsid: Kernel reported iSCSI connection 1:0 error (1022 - ISCSI_ERR_NOP_TIMEDOUT: A NOP has timed out) state (3)
Apr 03 05:00:55 manjaro-laptop iscsid[967]: iscsid: connection1:0 cannot make a connection to 192.168.1.101:3260 (-1,101)
Apr 03 05:00:58 manjaro-laptop iscsid[967]: iscsid: connection1:0 cannot make a connection to 192.168.1.101:3260 (-1,101)
Apr 03 05:01:01 manjaro-laptop iscsid[967]: iscsid: connection1:0 cannot make a connection to 192.168.1.101:3260 (-1,101)
Apr 03 05:01:04 manjaro-laptop iscsid[967]: iscsid: connection1:0 cannot make a connection to 192.168.1.101:3260 (-1,101)
Apr 03 05:01:07 manjaro-laptop iscsid[967]: iscsid: connection1:0 cannot make a connection to 192.168.1.101:3260 (-1,101)
Apr 03 05:01:10 manjaro-laptop iscsid[967]: iscsid: connection1:0 cannot make a connection to 192.168.1.101:3260 (-1,101)
Apr 03 05:01:13 manjaro-laptop iscsid[967]: iscsid: connection1:0 cannot make a connection to 192.168.1.101:3260 (-1,101)
Apr 03 05:01:16 manjaro-laptop iscsid[967]: iscsid: connection1:0 cannot make a connection to 192.168.1.101:3260 (-1,101)
Apr 03 05:01:19 manjaro-laptop iscsid[967]: iscsid: connection1:0 cannot make a connection to 192.168.1.101:3260 (-1,101)
Apr 03 05:01:22 manjaro-laptop iscsid[967]: iscsid: connection1:0 cannot make a connection to 192.168.1.101:3260 (-1,101)
Apr 03 05:01:25 manjaro-laptop iscsid[967]: iscsid: connection1:0 cannot make a connection to 192.168.1.101:3260 (-1,101)
Apr 03 05:01:28 manjaro-laptop iscsid[967]: iscsid: connection1:0 cannot make a connection to 192.168.1.101:3260 (-1,101)
Apr 03 05:01:31 manjaro-laptop iscsid[967]: iscsid: connection1:0 cannot make a connection to 192.168.1.101:3260 (-1,101)

I missed that part. I just had a Synology box behind me, and said, “Hey, let’s try it.” Since no one else was answering, I thought I might be able get you on the right path. I mainly used iSCSI on SPARCs, DEC Alphas, and others; well before x86/amd64. So this is all pretty much new to me too, as they did not run Linux. Getting it working before moving into a configuration file, or whatever the case, is often the preferred route. iSCSI is a little weird in this respect.

It was also right before bed when I attempted it, just like I’m doing now now. But after almost two hours. I give up, at least for today.

I can get everything to persist and work upon reboot, except the actual login command.

You probably have figured out that open-iscsi stores data in records. You can view or edit them right from /var/lib/iscsi/nodes/.../... But the smart way is to iscsiadm -m node -T ICQ -o update -n property -v value Even with everything I can find like node.startup, node.conn[0].startup, etc. It all saves and persists across reboots, but I still have to run the iscsiadm login command. Even just iscsiadm -m node -L all alone after rebooting makes it all work for me.

Making this a unit/startup-script could be a work around. I hate it, but I’ve been searching and trying a lot of things to avoid that. I’ve found dozens of people asking the same question specifically about the login part. And now it’s wayyyyy past my bedtime.

Also, I did not get your first errors, but I did get a bunch of cannot make a connection errors. But it was only to the IPv6 address I didn’t have configured, and it timed out after one min of trying. At least for me, that would be an easy fix (just to remove the IPv6 node). How many nodes do you have?

e.g.

iscsiadm -m node
[NAS]:3260,4294967295 iqn.2000-01.com.synology:nas.Target-1.24f9266b17c
[fe80::9209:d0ff:fe3c:80cd]:3260,1 iqn.2000-01.com.synology:nas.Target-1.24f9266b17c
sudo iscsiadm -m node
10.8.0.1:3260,1 iqn.2000-01.com.synology:218plus.Target-1.80e73d78a6c
192.168.1.101:3260,1 iqn.2000-01.com.synology:218plus.Target-1.80e73d78a6c
10.8.0.1:3260,1 iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c
192.168.1.101:3260,1 iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c

I guess I am just gonna go the

road as I have no idea how to fix this issue directly. Is this a bug in Arch/Manjaro?

Also on boot and shutdown it is still trying unsuccessfully to connect to iscsi.

How do I fix that?

I have created a bash file and added it as startup application in the UI:

#!/usr/bin/env bash

sudo iscsiadm -m node --target iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c --portal 192.168.1.101:3260 --login
sudo mount -a

When i run the file from the Terminal it works, but the Startup script doesnt, I rebooted and iscsi was not mounted after booting.

Also even though I changed /etc/iscsi/iscsid.conf again to boot mode manual, but booting and shutdown still takes a very long time and I get the iscsi errorsmessage on boot as before.

What else do I need to set, so it doesnt try to connect to iscsi anymore on boot?

Edit: I checked some more logs:

sudo journalctl -b |grep -i iscsi

Apr 03 16:15:40 manjaro-laptop systemd[1]: One time configuration for iscsi.service was skipped because of an unmet condition check (ConditionPathExists=!/etc/iscsi/initiatorname.iscsi).
Apr 03 16:15:53 manjaro-laptop systemd[1]: Listening on Open-iSCSI iscsid Socket.
Apr 03 16:15:59 manjaro-laptop systemd[1]: Starting Open-iSCSI...
Apr 03 16:15:59 manjaro-laptop kernel: Loading iSCSI transport class v2.0-870.
Apr 03 16:15:59 manjaro-laptop systemd[1]: Started Open-iSCSI.
Apr 03 16:17:29 manjaro-laptop systemd[1]: Dependency failed for /mnt/iscsi-vmware.
Apr 03 16:17:29 manjaro-laptop systemd[1]: mnt-iscsi\x2dvmware.mount: Job mnt-iscsi\x2dvmware.mount/start failed with result 'dependency'.

So maybe the issue is just I didnt rename the /etc/iscsi/*.iscsi file correctly?

sudo ls -l /etc/iscsi/
total 20
-rw-r--r-- 1 root root    80 26. Mär 03:31 initiatorname.iscsi
-rw-r--r-- 1 root root 14047  3. Apr 16:03 iscsid.conf

I guess initiatorname.iscsi is not correct?
What do I need to name it?

The contents of the file are correct though:

InitiatorName=iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c

I gave this another couple hours trying to make it work, I really thought I was getting close. Maybe I’ll just post this for now, but I haven’t given up yet. I just need some free time.

I enabled CHAP thinking it might help, since the login part is the issue, but nope. (CHAP works fine though, just need the manual login command.)

And seeing this in the man page for iscsiadm:

-L, --loginall==[all|manual|automatic|onboot]
    For node mode, login all sessions with the node or conn startup values passed in or all running session, except ones marked onboot, if all is passed in. 

Trying everything to set it to onboot just comes up with invalid record. (At least in node mode.)

What I didn’t do anything with, is the other modes (like iface). (Node mode seems like what we need!)

It’s safe to say, it’s not a bug with Arch or Manjaro. Like I said, I’m setting this up for the first time too. I’ve done a lot of work with iSCSI, but my decade old experience with iSCSI on SANs (storage area network) isn’t even helping that much here. What I do know, is that it can be finicky to setup initially. (And iSCSI isn’t common in the home and small business sector.)

As for making this a startup script. I won’t go into detail because this is just the same thing for any Linux distro using systemd, which is the majority of the popular ones. (Create a simple systemd service unit file and run a script on boot | Support | SUSE)

But you would probably want to out take the mount -a of your script. A mount unit is designed for things like this.

Edit: I forgot about the timeout. I can come to that, but the quick answer is setting stricter timeouts, but you have to do that per node, and multiple values unfortunately. And I would put a stricter kill timeout right in the iscsid.service, but only after you know the other ones work first.

I wish you a lot of free time to help me, because id really like to get this issue resolved, thanks for helping me. :slight_smile:

Whats a mount unit?

You mean instead of using mount -a I should just mount the iscsi mountpoint?
I just changed that to only mount that one by UUID.

The mount -a was more or less just for testing.

I have now deleted the other three targets in iscsiadm -m node, which I found by another Google Search.

Node now only finds the target I need.

Well then I was googling how to change to login timout.

I did it like described here:

basically with
sudo iscsiadm -m node -T iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c -o update -n node.conn[0].timeo.login_timeout -v 1

I thought this would change the timeout on my Client, but this seems to have changed the timeout in the LUN settings.

Well this was dumb as it turns out, because now I can no longer login to the LUN:

sudo iscsiadm -m node --target iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c --portal 192.168.1.101:3260 --login
Logging in to [iface: default, target: iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c, portal: 192.168.1.101,3260]
iscsiadm: Could not login to [iface: default, target: iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c, portal: 192.168.1.101,3260].
iscsiadm: initiator reported error (8 - connection timed out)
iscsiadm: Could not log into all portals

I checked the settings on the NAS, but it seems I can not change this setting in the SAN Manager.

How do I revert this change via SSH on the Synology NAS?

Edit: Well I have found the actual correct setting in /etc/iscsi/iscsid.conf :
node.session.initial_login_retry_max = 3
Now booting and shutdown times are normal for Manajaro at least.

Edit2: after another reboot I can manually login to iscsi again. But mounting on boot still doesnt work.

Checking the settings in the LUN with
sudo iscsiadm -m node --target iqn.2000-01.com.synology:218plus.default-target.80e73d78a6c --portal 192.168.1.101:3260

I see these (hopefully) relevant settings:

node.startup = automatic
node.leading_login = No
node.conn[0].address = 192.168.1.101
node.conn[0].port = 3260
node.conn[0].startup = manual
node.conn[0].tcp.window_size = 524288
node.conn[0].tcp.type_of_service = 0
node.conn[0].timeo.logout_timeout = 15
node.conn[0].timeo.login_timeout = 15
node.conn[0].timeo.auth_timeout = 45
node.conn[0].timeo.noop_out_interval = 5
node.conn[0].timeo.noop_out_timeout = 5
node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
node.conn[0].iscsi.HeaderDigest = None
node.conn[0].iscsi.DataDigest = None
node.conn[0].iscsi.IFMarker = No
node.conn[0].iscsi.OFMarker = No

Do need to set
node.conn[0].startup = manual
to automatic?

is node.conn[0] the setting on my client or the NAS side?
Or is this just set to manual, because I am currently manually connected to the LUN?

When you are mounting it that way, with -a or not, I assume you have this hard coded in /etc/fstab.

I don’t want to dive too deep into this, really most of what you need is common systemd knowledge you can find anywhere.

But for a quick example, if your config was in a file in such as: /etc/systemd/system/iscsi.mount
You can pick a better name than iscsi, but that it ends in .mount is crucial as it is a mount unit. But in that file, you can do things like use After=network.target & Requires=iscsid.service, or not even contacting the LUN until it’s used.

Yes, you want to clean up all these failed attempts. Nodes, sessions, etc. I was using find /var/lib/iscsi/ to check if I got them all.

With care, you can view and edit all of the settings (at least the ones you set via -n -v by editing: /var/lib/iscsi/nodes/.../.../default But edit at your own risk, the formatting must be exact.

(iSCSI is popular in cases where a network card boots off a LUN, so this configuration setup is also designed to also be sent to a NIC’s firmware.)

You can’t change settings on a iSCSI target, aka your Synology box, this way anyway; everything on their end, for configuration do only through their DSM interface. After creating the LUN, you shouldn’t need to touch it again. The one exception would be in SAN Manager/Hosts, you should see one initiator after you ran the iscsiadm login command.

Back to cleaning stuff up, and it looks like you have something trying to login via iface mode as opposed to node.

Fake edit: I see your edit.

When searching that setting, you can see many references to an old NetApp doc. (They make enterprise NAS much much more powerful than our little Synology boxes). They are very insistent that you need that setting. So I set it to automatic, it’s not breaking anything. And if it did, we would likely see it complain.

All these settings are client side (initiator in iSCSIeez), saved by iscsid in /var/lib/iscsi.

I have taken so long to respond to all this, I didn’t even get to try next move. Tomorrow. We can only hope.

Hey,

hope you had some time to do more testing.

For some reason the booting and shutdown now again takes as long as before (2mins) with the iscsi errormessage on boot.

Although node.session.initial_login_retry_max is still 3 in /etc/iscsi/iscsid.conf

Also the bash script to mount iscsi for some reason does not work in autostart.
But it works, when I run it manually. Any idea why?

On a more or less unrelated note:

How do I increase the sudo timeout for the graphical sudo password question?

So I think found the problem. I guess I just needed to look at this a different day a different way with more sleep.

In addition to enabling the iscsid service, you also need to enable the iscsi service. This runs the login command.

So great, right? The problem with the default configuration in the iscsi.service file throws an error.

In /usr/lib/systemd/system/iscsi.service, it has the line:

ExecStart=/usr/bin//iscsiadm -m node --loginall=automatic -W

But you don’t want to modify files here. So to edit this you do a systemctl edit iscsi, then change the above line to:

ExecStart=/usr/bin//iscsiadm -m node --loginall=all

(Your unit configuration changes gets stored in /etc/systemd/system/iscsi.service.d/override.conf)

Then everything starts up fine, and logs in automatically for me.

I just did that and rebooted, but its still not working for me.

When I edited the file it says

But /usr/lib/systemd/system/iscsi.service still uses the old configuration:

Also the file you mentioned does not exist for me:

The changes are in this file though:

And journalctl logs these errors:

Edit: Status of iscsi also gives me an error:

Edit2:
sudo systemctl edit iscsi looks like this, is this correct?

Nevermind, I just didnt have the iscsi service enabled, now it finally works.

Thanks a lot! :slight_smile:

But I am still getting this error:

Apr 10 20:03:47 manjaro-laptop systemd[1]: /etc/systemd/system/iscsi.service.d/override.conf:1: Assignment outside of section. Ignoring.

Do I need to fix this, now that it works?

Edit:
I noticed iscs service is still using the old command:

ExecStart=/usr/bin//iscsiadm -m node --loginall=automatic -W

So for some other reason its working now.

No, hence you see errors:

The ExecStart=.. line belongs into [Service] section. It should look like this:

### Editing /etc/systemd/system/iscsi.service.d/override.conf
### Anything between here and the comment below will become the contents of the drop-in file
[Service]
ExecStart=
ExecStart=/usr/bin//iscsiadm -m node --loginall=all

### Edits below this comment will be discarded


### /usr/lib/systemd/system/iscsi.service
# [Unit]
# Description=Login and scanning of iSCSI devices
# Documentation=man:iscsiadm(8) man:iscsid(8)
# Before=remote-fs.target
# After=network-online.target iscsid.service
# Requires=iscsid.socket iscsi-init.service
# Wants=network-online.target
#
# [Service]
# Type=oneshot
# ExecStart=/usr/bin//iscsiadm -m node --loginall=automatic -W
# ExecStop=/usr/bin//iscsiadm -m node --logoutall=automatic
# ExecStop=/usr/bin//iscsiadm -m node --logoutall=manual
# SuccessExitStatus=21 15
# RemainAfterExit=true
#
# [Install]
# WantedBy=remote-fs.target

However if it’s working now you might not need the override at all.

It won’t be running because it calls the login command and exits.

It is a strange approach, which is why I was baffled for so long. But I completely missed seeing that file my first time around. (With having them together, iscsi.service, being next to iscsid.service, and the other iscsi* files, I ended up glancing over it.)

As for the rest? That’s a lot of backporting edits to know what stage you’re at now. I can now set this it up in a minute or two (now, anyway), did you clean stuff up?

And overriding systemd units is another easy thing to look up, and I’m sure there’s people much better than I here familiar with them. But I must stress that don’t want to mess this part up. Just because it’s working now, you could have an open-iscsi update down the road botch everything.

(But I am fairly certain it will not work using the Manjaro package default, just that one line. It will not login to a configured node anyway.)

Yes, lets continue rolling back the changes.

I just had a very weird thing happen to me:

I was no longer able to login as root. It always said my password has changed.

I thought it was because of a conky script I executed or that I changed the Manjaro hostname and was already panicking as I also couldnt rollback any changes with timeshift since it requires su privileges.

Now I rebooted again and my old password works again, whats an explanation for that?

Redarding the two iscsi services: thats what I suspected in my first post :smiley:

I did not clean up anything for now, because im just glad iscsi on boot now finally works and im afraid it will no longer work if I change anything, just like you :smiley:

Aynways, thanks for sticking with me with this issue for so long.
I think I never had anyone in any other forum or discord help me so consistent and for over 2 weeks with an issue- Usuallz they give up after one day, just like I would have.

Many people wrote to me I shouldnt use Manjaro because there were some misconfigured yay package versions, that DDoSed the Arch servers and some other things, but this positive exeperience with you really makes me hopeful.

Edit: Its getting weirder, after the reboot it again said my password for sudo and grpahical sudo is wrong.
Now after rebooting again I can login again as su?!

Should I just reset Manjaro with timeshift to before 2 days?
But then iscsi wouldnt work again I guess?

The most important question I guess:

Whats a good way to backup everything in Manjaro, so that if this problem should persist I can just reinstall and restore everything from the Backup so its runs like now without the root issues ofc?

Edit2: Nevermind it was that conky script I wrote. Disabled it and now I can login as root again every time.

It was an awkward time where I couldn’t VPN home (where my NAS was). But I’m glad we got it solved. Maybe flag that post as the solution, where I started with: “So I think found the problem.” If I knew that first, it would have taken me a small fraction of the time to figure out, with just knowing that piece of information there.

There have been many questions and tangents, but I tried to stay focused on the task at hand! Feel free to start another topic about it and tag me in it, and I can tell you what I do. And I’m sure people with more knowledge than me, will either beat me to it, or have something better than what I do.

I’m not affiliated with Manjaro in anyway, I just use it. And those reasons aren’t even logical.

For one, even some of the biggest organisations in the world have had this problem on more than one occasion. And the Manjaro Foundation was, and is mostly volunteers. It is expensive, and sometimes impossible, to protect against a DDoS attack with enough firepower.

With yay and versioning, this sounds all related to AURs. If you are blindingly installing AURs, yes, you will have a bad time.

The biggest downfall I can think of, is that does have a slightly higher starting learning curve to use (compared to some of the other popular distros). Myself, I like those extra steps, since I handle it my own way. And caveats come with any rolling release distro. (But I think Manjaro does makes this learning curve smaller.)

Actually that wasnt the solution.

I wrote two posts above, that it still uses
ExecStart=/usr/bin//iscsiadm -m node --loginall=automatic -W

and works now for some different reason and I dont really know why.

But ill tag it anyways as solution and open a new topic about the backups.

I know youre not affiliated with Manjaro, I didnt mean this was a good experience with the official Manjaro Team, but with the Manjaro Community, which youre part of.

Disable the service and reboot, or just logout; then type that command. You will see it fail.

You can see what it is using with the command:

sudo systemctl show iscsi.service --property=ExecStart
ExecStart={ path=/usr/bin/iscsiadm ; argv[]=/usr/bin//iscsiadm -m node --loginall=automatic -W ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }
ExecStart={ path=/usr/bin/iscsiadm ; argv[]=/usr/bin/iscsiadm -m node -L all ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }

We see the /usr/lib one first, then the one it’s using in /etc, as it’s being overridden.

To elaborate some more… You can override the entire file (with using edit --full), or just that one property; I chose the former, which makes sense for this. This isn’t just open-iscsi related, but If any package gets updated that uses systemd units, they often modify these unit/service files. So you get all the new changes, except for the single property (line) that could break it (for you).

When you do the sudo systemctl edit iscsi.service (or as I rushed to get down before: systemctl edit iscsi).

The file tells you where to put the changes, as so:

### Editing /etc/systemd/system/iscsi.service.d/override.conf
### Anything between here and the comment below will become the contents of the drop-in file


### Edits below this comment will be discarded
 ...

So in between, or starting on line 3, you add:

[Service]
ExecStart=/usr/bin/iscsiadm -m node -L all

Once you save the file, you can see just those two lines in: /etc/systemd/system/iscsi.service.d/override.conf it made for us.

If you did modify the /usr/lib/systemd/system/iscsi.service file, it would work. But the next time open-iscsi is updated, it would break. This is part of the reason I was stressing about getting this part right.

The suggestion of making that post the solution was for other people, that probably know their way around systemd (or can at least web search their way through it). I configured a node and enabled it with ease, and then I got stuck. If I knew about the iscsi.service (not iscsid.service), and how they chose to use it by hard coding the login method, it would of taken me minutes, as opposed to hours. (But as I said, I did glance over that file I needed to see, when I did a pacman -Ql open-scsi the first day! I blame lack of sleep. Uhh… Yeah.)