How to process files from a different computer?

I want to run this program on a folder from another computer. For that I’ve shared an NFS folder between two computers with manjaro.

In the other computer I’ve done:

❯ sudo mount -t nfs 192.168.1.122:/X /data/nfs/X
❯ ls -l /data/nfs/X
works
❯ python main.py folder "/data/nfs/X"
[INFO] get_files_from(/data/nfs/X)
doesn't work

The program only shows a log output and then closes.

Resources

With Linux everything is a file - conceptual.

So if you have mounted a remote folder on a local mount point as you describe with the console output and your python script (app) doesn’t work as expected then it is a coding issue within your python code.

If your python script (app) fails due to write access it is a permission issue with your local mount point.

The remote permissions matter but they are overridden by permissions on the local mount point.

2 Likes

I’m no expert by any means, but that, that looks like a permissions error. So check your permissions and mount.

Edit:

I’ve removed the Solution mark, as this :point_up_2: isn’t the/a solution.

I’ve seen that the folder is owned by root. But it won’t allow changing the owner:

sudo chown $USER:$USER /data/nfs/X
chmod: change owner '/data/nfs/X': not permitted

I’m not sure, but that might be the problem. So I’d recommend getting your mount points fine first:

Because, the might be it…

unmount the mount point - change permissions

sudo chmod ugo+rwx /data/nfs/X

Then check your export - rw otherwise you will fail.

How did you set /etc/exports ?

Assuming the source of the script is not just a random piece you find useful - there is a lot hard coded paths which likely will cause issues unless you refactor to fit your exact use case.

And no - I won’t help with that.

I’ve repeated the process with another drive and adding that instruction but I still see everything owned by root.

server ❯ sudo mkdir -p /mnt/V
server ❯ sudo chmod ugo+rwx /mnt/V
server ❯ sudo umount /run/media/user/V
server ❯ lsblk -no UUID /dev/sdc1
4435D137344DB3E9
server ❯ sudo cat /etc/systemd/system/mnt-V.mount
[Unit]
Description=My shared partition

[Mount]
What=/dev/disk/by-uuid/4435D137344DB3E9
Where=/mnt/V
Type=ntfs
Options=defaults,rw,noatime

[Install]
WantedBy=multi-user.target
server ❯ sudo cat /etc/systemd/system/mnt-V.automount
[Unit]
Description=Automount shared partition
ConditionPathExists=/mnt/V

[Automount]
Where=/mnt/V
TimeoutIdleSec=10

[Install]
WantedBy=multi-user.target
server ❯ sudo systemctl enable --now mnt-V.automount
server ❯ ls -l /mnt
drwxrwxrwx 1 root root 4096 jun 28 18:10 V
server ❯ ls -l /mnt/V
everything is owned by root
server ❯ sudo cat /etc/fstab
/mnt/V /srv/nfs/V none bind 0 0
server ❯ sudo mkdir -p /srv/nfs/V
server ❯ systemctl daemon-reload
server ❯ sudo mount -a
server ❯ ls -l /srv/nfs
drwxr-xr-x 2 root root 4096 jul 17 15:11 V

Seriously? NTFS don’t save Linux permissions on its filesystem. You have to set it as a option when mounting.

2 Likes

I’ve followed this tutorial to share a folder with NFS but maybe because the folder is a NTFS partition all files are owned by root and I can’t use them.

This is what I’ve done, I need to fix whatever I’ve done wrong.

server ❯ sudo mkdir -p /mnt/V
server ❯ sudo chmod ugo+rwx /mnt/V
server ❯ sudo umount /run/media/user/V
server ❯ lsblk -no UUID /dev/sdc1
4435D137344DB3E9
server ❯ sudo cat /etc/systemd/system/mnt-V.mount
[Unit]
Description=My shared partition

[Mount]
What=/dev/disk/by-uuid/4435D137344DB3E9
Where=/mnt/V
Type=ntfs
Options=defaults,rw,noatime

[Install]
WantedBy=multi-user.target
server ❯ sudo cat /etc/systemd/system/mnt-V.automount
[Unit]
Description=Automount shared partition
ConditionPathExists=/mnt/V

[Automount]
Where=/mnt/V
TimeoutIdleSec=10

[Install]
WantedBy=multi-user.target
server ❯ sudo systemctl enable --now mnt-V.automount
server ❯ ls -l /mnt
drwxrwxrwx 1 root root 4096 jun 28 18:10 V
server ❯ ls -l /mnt/V
everything is owned by root
server ❯ sudo cat /etc/fstab
/mnt/V /srv/nfs/V none bind 0 0
server ❯ sudo mkdir -p /srv/nfs/V
server ❯ systemctl daemon-reload
server ❯ sudo mount -a
server ❯ ls -l /srv/nfs
drwxr-xr-x 2 root root 4096 jul 17 15:11 V
server ❯ sudo cat /etc/exports
/srv/nfs            192.168.1.0/24(rw,sync,crossmnt,fsid=0)
/srv/nfs/V          192.168.1.0/24(rw,sync)
server ❯ systemctl enable --now nfs-server.service
client ❯ systemctl enable --now nfs-client.target
client ❯ sudo mkdir -p /mnt/V
client ❯ sudo mount -t nfs 192.168.1.122:/V /mnt/V
client ❯ ls -l /mnt
drwxr-xr-x 2 root root 4096 jul 17 15:11 V

Additionally set Owner and Group:

uid=1000,gid=1000

Set permissions:

dmask=0002,fmask=0113
  • files: -rw-rw-r--
  • dirs: drwxrwxr-x

Maybe this helps also:

windows_names,hide_hid_files,hide_dot_files,big_writes

Note that NFS expect the same UID on client and server for writing to it. You have to set permissions for other, so that everyone can write if that is not the case or use usermapping.

4 Likes

Where does this go? In /etc/systemd/system/mnt-V.mount: Options=defaults,rw,noatime,uid=1000,gid=1000,dmask=0002,fmask=0113,windows_names,hide_hid_files,hide_dot_files,big_writes?

So by copy pasting those values it will work if the UIDs are different?

After changing /etc/systemd/system/mnt-V.mount I’ve done

sudo umount /mnt/V
sudo systemctl start mnt-V.automount
❯ ls -l /mnt/
drwxr-xr-x 2 root root 0 jul 17 16:02 V

So the permissions have changed but the ownership that is what is giving me trouble is the same.

I quoted the part…

No… check the UID and change it: id on the client.

The permissions of the folder will not change, but the content of the mountpoint.

Ok I saw it in three parts so I thought only the first part went with the quoted text.

Well I’ve checked the UIDs and they are the same so I’ll be sure never to create another user just so I don’t have to complicate this any further. I don’t understand why it doesn’t just share it with all users instead of having to be checking the user id. As if it wasn’t difficult enough already.

Check: man exports and search for anonuid and anongid. Maybe this is what you want.

server ❯ sudo cat /etc/systemd/system/mnt-V.mount
[Unit]
Description=My shared partition

[Mount]
What=/dev/disk/by-uuid/4435D137344DB3E9
Where=/mnt/V
Type=ntfs
Options=defaults,rw,noatime,uid=1000,gid=1000,dmask=0002,fmask=0113,windows_names,hide_hid_files,hide_dot_files,big_writes

[Install]
WantedBy=multi-user.target

server ❯ sudo umount /mnt/V
server ❯ sudo systemctl start mnt-V.automount
server ❯ ls -l /mnt/V
drwxrwxrwx 1 root root 4096 jun  5 19:49 'System Volume Information'
client ❯ sudo mount -t nfs 192.168.1.122:/V /mnt/V
client ❯ ls -l /mnt/V
total 0

Give you a favor and use a linux filesystem with NFS. Like ext4. NTFS is by all mean not tested with it.

1 Like

I have a hard time understanding why you cannot get it to work - I even began questioning my guide.

Now I can’t have that - so I went over to the guide - the guide I wrote many moons ago - I went through it step by step.

To have something to work with - you gotta love those rpi boards - I created a sdcard with a minimal manjaro arm.

Booted the pi - initial oem setup aside - I took my test USB - created earlier today - NTFS formatted and all.

I followed my guide to the letter

  • setting up a mount unit
  • mounting the usb by uuid
  • checked the mount point - checked it was writable (as it is ntfs there is no point in setting permissions)
  • enable nfs-server.service
  • modify /etc/exports (nfs v4)
  • created a service folder /srv/nfs/ntfs
  • did a bind mount

Over to my workstation

  • enable nfs-client.target
  • create a mount point
  • took a shortcut
  • only manual mount in a temp folder in my home
  • but it worked
  • listed existing files - test I created earlier on the pi
  • created a new file

So I realised - the guide is OK - if you follow it to the letter - it works - and as it is ntfs - which Linux do - permission wise - treat like an exfat system - world read-write - even when shared across computers using NFS.

1 Like

I’ve decided to use an ext4 hard drive so I’ll restart the process in a few weeks. Meanwhile I would like to leave the system as it was. But I’m having trouble: Drive changed name after reverting changes of sharing data using NFS