[CIFS/SMB] zero-padded files after download

[mayly]

after downloading from a windows share to my raspberry pi 4 on manjaro aarch64 (using kde) i have seen that the copied files got padded with zeros up to 64kB.

• when i tried this on raspberry pi os (32 bit debian) with xfce’s thunar the same happened, but the files have been padded to 256kB there. when i cp’d from the /run/user/1000/gvfs/smb-share... directory in terminal the file had correct size. when i did the same in thunar (from the same locally mounted directory) it got blown up with zeroes again. double commander listed the opened share in the drive list already, accessing it as smb://... – copying caused the same zero padding. but when doublecmd accessed /run/user/1000/gvfs/smb-share... as a directory, without using the drive list, the files copied normally.

it seems like non-fuse-gvfs and kio show this bug, but fuse-gvfs does not. is cifs/smb broken in general or just on arm?

[linux-aarhus]

@mayly

is cifs/smb broken in general

cifs and smb is one and the same thing with SMB the official name.

Samba is a reverse engineering of Microsofts proprietary SMB.

What you are calling zero paded is a common padding char[0] or null terminated string.

The differences is likely due to packet size during network transfer.

What you see may be different stages of the files life in transfer vs. being read from a permanent mountpoint

@mayly

show this bug,

→ Is it a bug? I don’t know.
→ Is it cause to concern as of data integrity? Perhaps.

Generally speaking the null char - what is displayed as 00 in hexadecimal notation is a common terminator used on lowlevel c programming language to signal the end of a data-stream and should be regarded as safe.

You may be able to conduct futher investigation if you create a temporay mount supply credentials from a file

 $ cat credentials
username=
password=
domain=

sudo mount -t cifs -o credentials=<credentials>  //server/share /mnt

Or supplying on cli

sudo mount -t cifs -o username=user,password=pwd,domain=domain //server/share /mnt

Technically gvfs and kio are wrappers around smbclient which is - kind’a ftp but for Windows network file servers.

Then compare the two results

→ using a file manager (layered smbclient)
→ direct access using a mountpoint

[mayly]

@linux-aarhus

What you are calling zero paded is a common padding char[0] or null terminated string.

that’s how the c language handles strings (rust btw uses pointer+length encoding instead, so there’s no universal rule). the observed problem on the other hand involves every type of data getting extended with zero-bytes up to the 64kB boundary.

@linux-aarhus

The differences is likely due to packet size during network transfer.

What you see may be different stages of the files life in transfer vs. being read from a permanent mountpoint

i had similar thoughts, suspecting the data being read into a 64kB buffer (on manjaro aarch64; on raspberry pi os 32-bit-arm it’s even 256kB) but the file doesn’t get truncated for some reason at EOF.

@linux-aarhus

→ Is it a bug? I don’t know.
→ Is it cause to concern as of data integrity? Perhaps.

yes, those are essentially different files now. i first noticed it when copying a source code repository and its size increased tenfold. most checksums won’t match and, depending on the file type, some binary files cannot be opened anymore because they’re corrupted with zeroes at their end.

@linux-aarhus

→ direct access using a mountpoint

yes, the kernel driver (requires root and knowledge) works correctly, as does gvfs using the fuse mountpoint. what does not are kio and gvfs (via file manager api, not the fuse mountpoint).