Having a strange and obscure issue happening with Dolphin/KDE and Network Drives!

Support KDE Plasma
,
1

Hello, as the title says, I’m having a strange issue with what I think is the Dolphin file manager and a folder on a network drive share. I’ve tried everything, and did a good bit of searching, but I can’t seem to find answers. Also, I’m not even certain how to properly frame my situation or questions to get answers.

Okay, let me give you the environment:

I have a local server with a proxmox hypervisor setup, and it’s running three VMs, which are TrueNAS, PiHole and Manjaro.

My Manjaro environment is KDE Plasma (x11).

Basically, I use the Manjaro VM for various catch-all functions. I have a VPN setup, so that I can view and test site’s I’m working on, and to also use various apps and functionality not present on my main workstation, which is a mac. I access the Manjaro VM using a VNC connection from my mac or console within proxmox.

So far, this setup works fantastically, with no issue…UNTIL this past weekend.

I have a NAS share mounted to a folder located in /mnt. Let’s call the main parent share “NASDrive”. Within that folder, I have about a dozen other folders. Let’s say one of which is named “vids”, which contains 1,410 other subfolders.

AND NOW WE GET TO IT!:

So, I go to access “NASDrive/vids”, as I’m want to copy some files to another drive, as I want to work on the videos contained therein. Well, the copy process also Dolphin itself freeze up, completely. I have to use the task manager to quit dolphin, and so, I decide to restart the VM entirely.

Manjaro restarts just fine with no issue. The KDE desktop loads just fine. So, I click to open Dolphin and go back to try and copy my videos again…BUT, the folder “vids” is only showing SOME, BUT NOT ALL of my files contained in that folder. It’s only shows about 115 to 119 of my subfolders, not the full 1,410 that should be there. Every single other folder on my network share is perfectly fine, no issues at all. The only folder not showing the full contents is the “vids” folder.

But there’s more! If I add a file of any kind to “vids”, lets just say a blank text file (text.txt), it will correct itself! Suddenly, I can see and interact with all 1,410 subfolders…UNTIL, I restart Dophin. If I close Dolphin and reopen, it goes back to only showing me 115-119 files again. I have no idea why it only shows me that many subfolders and why it changes between 115 - 119? But again, if I just add in any folder to “vids” it will suddenly correct itself, albeit temporarily.

I’VE TRIED EVERYTHING I CAN THINK OF!

I’ve unmounted the NAS drive, I’ve deleted and recreated the “NASDrive” mounted folder within /mnt, I’ve created another folder named “vids2”, and moved everything from “vids” into “vids2” and then renamed it (I even tried this twice, once from within Manjaro and then again from my mac), I’ve created another folder within /mnt and mounted the NAS drive to it…NOPE, nothing seems to work. When I close and then reopen dolphin, and view “vids”, it only shows between 115 - 119 of the folders. I have to keep pasting in a file to cause it to show the full contents. Also, bear in mind, I’m shutting down or restarting the VM each time I try something to fix it.

I can 100% say that the NAS server is working fine. I have no other issues accessing the “vids” folder from any other machine at my location at all. All content of the “vids” folder are just as they should be.

ALSO, when I pull up terminal and navigate directly to “/mnt/NASDrive/vids” and do a ls and view the directory listings, NO PROBLEMS AT ALL. Terminal lists all of the files content just fine, so it only appears to be an issue in dolphin. HOWEVER, if I open Dolphin and enter a NAS path directly in the path bar, at the top, like so:

smb://10.1.1.18/NASDrive/vids

That also works just fine, with no issue! It’s just unreal at this point! For whatever reason, this only seems to be an issue within Dolphin and via a mounted network drive from within /mnt, and it only effects the one specific “vids” folder!

And again, all of this all started happening this past weekend when I was copying fils from “vids” into another folder. It just suddenly froze and I had to end Dolphin in the task manager and restart the VM.

I’m NOT a linux guru by any stretch and any help would be GREATLY appreciated! I have no clue at this point. And again, it’s such a strange issue, I’m not sure how to properly frame the questions to find answers.

Is there some kind of cache somewhere that I need to dump? is there a corrupted file or folder I need to root out? Is there something somewhere I need to reset? Again, any help would be appreciated!

Thank you!

2

Have you tried Krusader instead of Dolphin or any other GUI file manager?
Have you tried to rename those file and folders:

~/.config/dolphinrc
~/.local/share/dolphin
~/.cache/dolphin
3

Yes, I’ve tried all of those items you’ve highlighted, but it didn’t correct the issue. Thank you for the suggestions though.

I’ve renamed all of those files/folders and then dumped everything in the .cache folder via command line, and then reopened Dolphin and it still returns to the same behavior.

I installed Krusader and when I opened that up, it worked like a charm. However, that didn’t surprise me because I knew the issue must be something related to Dolphin, because command line file navigation works fine. So, It’s not a problem with the share, smb or TrueNAS. It works fine accessing from my other machine.

Something about having to force dolphin to quit and then forcing the VM to restart has effected Dolphin in some way that I can’t pin down.

4

I don’t know if this comment can solve the problem, but Dolphin’s configuration file is as follows.

Purpose Path
Dolphin settings dialog ~/.config/dolphinrc
Toolbar settings ~/.local/share/kxmlgui5/dolphin/dolphinui.rc
Sidebar “Place” settings ~/.local/share/user-places.xbel

Deleting “user-places.xbel” may improve the situation.
There are unknown bugs, such as bloat in this file.

5

Well, I made a backup of user-places.xbel, and then deleted the original, closed dolphin, and then reopened, and it unfortunately didn’t effect anything. It still only shows a small subset of files when first viewing the folder.

Also, to add a bit of supplemental information, which I just discovered, when doing a copy/paste (and only copy/paste, doing a cut/paste works fine so far) it will start to throw errors trying to copy and paste more than about 2 folder at a time. So, if I tried to copy, say 4 non-empty folders, to another folder on the same network drive, it give errors such as unable to copy, unable to make folders, would you like to retry and etc. Clicking retry will then give the overwrite dialogue box, and when I click overwrite, then it might work. However, if I attempt to copy, let’s say 10 or more folders, it will generally fail altogether. So, when copying files, I have to move them in small batches. Again , cutting and pasting works fine, apparently.

My TrueNAS VM has almost 26TB of storage, and I’m only using about 5TB total.

Also, and this I just noticed over the last few days, there is a disparity between reported RAM/Memory usage as reported in the Manjaro Task Manager versus the RAM/Memory usage as reported in the ProxMox administrator. The memory usage reported in the Manjaro task manager is usually around 2.8 - 2.8 GB, under normal daily usage situations. HOWEVER, the ProxMox admin reports that the Manjaro VM is using almost 13GB, over 90 percent of what I have allocated for that VM!? That’s a huge difference, and I have no idea whats causing that discrepancy?

6

Your memory usage reported by Manjaro is the usage known to the system inside the vm.

What happens outside is in charge of your host but if you see this in conjunction with accessing the network share it is likely caused by the host kernel caching or buffering the files consumed by the guest os.

As for your issue with the share - I assume you are using on-the-fly mounting supplied by dolphin and the kio subsystem.

That subsystem has over time proven to be unstable - to put it mildly.

I often read about users using /mnt for persistent mountpoints - even worse - I have even seen some try to use /run/ for persistent mounts.

Some advise from a lifetime sysadmin

  • don’t use /mnt
    • it is for temporary mounts
    • scripting often make use of /mnt
    • which may create weird issues when content is shadowed by a temporary mount
    • permissons on /mnt may interfere as this is usually root owned with read access for everyone
  • don’ use /home/$USER
    • permission issues you are not the only user
  • use a dedicated structure
    • short version could be /a/bla
  • use mount units
    • e.g. mount point is /a/bla
      • mount unit /etc/systemd/system/a-bla.mount
    [Unit]
Description=My bla mount
[Mount]
What=//<ip-address>/nasshare
Where=/a/bla
Type=cifs
Options=_netdev,rw,credentials=/etc/samba/bla-credentials
[Install]
WantedBy=multi-user.target

    sudo systemctl enable --now a-bla.mount
  • depending on your server’s capabilities you may need to add a samba version to Options=
    vers=3
  • adjust permissions on the /a/bla mount point as needed
    • permissions inside the share is controlled by the server
    • permissions on the mount point is controlled by the client
      • e.g.
        • add the current user to the group users
        • allow group users rw access to the mount point
          sudo chmod ug+rw /a/bla
      • or add file_mode=0777,dir_mode=0777 to the Options=
        • the actual result will depend on the privileges of user used to connect

More reading

7

Thanks for your post! Any help is appreciated!

Unfortunately, I followed you tip and created a mount unit. And, it all mounted correctly on the first attempt. However, visiting that same folder in question, with the issue, it still only shows me a partial file list. It shows the same 118 folders. BUT, for whatever reason, it does seem a bit better. Now, if I click another folder, and then back to the folder with the issue, after about 2.5 second, it will pop, and I’ll see the full file list for that folder.

BUT, how do I enable this mount unit methodology to mount the drive of system startup? I have to manually run the systemctl, specified in your post to get the drive mounted. What would be the best method to mount on boot?

This doesn’t 100% solve the issue, but it does help on the daily, thanks again.

8

I am assuming you are using samba - simply because it is the most commonly used share method.

Samba can be a complex system and even the simplest setups requires some level of understanding the concepts. It is never plug’n’play - more like plug’n’pray it works.

That is why I have written several guides on how to handle connecting to samba shares and how to share using samba on Manjaro.

If you are still having issues - you will have to look at your smb.conf - at the very least you must have an empty /etc/samba/smb.conf.

Previews of video and image files over a network file system will slow the system dramatically.

If I recall correct Dolphin wants to create a cached list of files and folders - this can take a long time with a remote file system of the size you describe - if you disrupt the process the cached list will not be complete - yet Dolphin think it is - thus it will only display what is known in the cache - you will have force a reread of the remote file system so dolphin can know what to expect when entering a folder.

With that in mind I think some of your issues is caused by impatience with dolphin processing the entire content of the remote filesystem.

enable --now argument means the unit is enabled and started immediately.

No you don’t - when a mount unit is enabled - it is started at boot - but - as the device is a network device the Options=_netdev will only mount when network becomes available and the target is available - not sooner.

You may tweak it by implementing an automount unit.

If you do implement an automount - before enable and starting the automount unit - you must disable and stop the mount unit - otherwise automount will fail.

All this is described in the topics I linked you to.

It is a network file system - and as such - you cannot expect a blistering fast experience like you can with a local file system. Also you are using Dolphin - and in my opinion there is far too many issues with it.

2.5s to list 118 folders - that is fast for a network file system.

On my systems I use pacmanfm-qt gvfs-smb gvfs-nfs as file manager - it is a so much better file manager compared to dolphin.

Usually there is no single multipurpose solution - network file systems is highly depending on the knowledge of those maintaining the file servers and consuming the data

This is not the share itself but a sub folder - so if you put that into the What= then you are going to have issues.

I could have stated in the unit - but I wanted to force you do some reading - a more thorough example

/etc/systemd/system/a-nasdrive.mount

[Unit]
Description=My NASDrive mount

[Mount]
What=//10.1.1.18/NASDrive
Where=/a/nasdrive
Type=cifs
Options=_netdev,rw,iocharset=utf8,file_mode=0777,dir_mode=0777,credentials=/etc/samba/bla-credentials,version=SMB3
TimeoutSec=30

[Install]
WantedBy=multi-user.target

The default samba version is SMB3
The TimeoutSec= is the amount of time elapsed before the mount attempt gives up.

Start the mount unit once - this will create the path with default permissions (root:root) - this may not be the desired permissions - so consider setting the correct permissions

sudo systemctl start a-nasdrive.mount
sudo systemctl stop a-nasdrive.mount

/etc/systemd/system/a-nasdrive.automount

[Unit]
Description=Automount NASDrive using samba
ConditionPathExists=/a/nasdrive

[Automount]
Where=/a/nasdrive
TimeoutIdleSec=300

[Install]
WantedBy=multi-user.target

The ConditionPathExists= ensures that you do not try to mount to a non-existing path
The TimeoutIdleSec= is the time elapsed before the file system is disconnected - there is technically no reason to keep the connection alive when it is not used.

sudo systemctl enable --now a-nasdrive.automount

The plus side of using automount for the remote file system - is that you cannot access the path if it is not mounted - thus you will never by accident shadow any content in the local folder - which is possible if the mount has failed due to an inaccessible service.

9

Thanks for the information and tips!

The mount unit I created was perfectly fine. Ultimately, it was a very simple issue with permissions. The subfolder in my custom structure, which I actually had the share mounted to, was root:root. I changed ownership to the current user and rebooted and that did the trick. It mounted on startup.

I did make a few tweaks, per your last post, with regard to option parameters, such as samba versioning, TimeoutSec and etc. But, I don’t think those tweaks had a noticeable effect. It was working yesterday with a manual intervention, just not on boot, which lead me to think it was related to permissions.

However, I’m kind of back where I started. With the mount unit approach, it’s slightly better than it was, with regard to file listing. But, at the same time, it’s almost the same. I can’t copy more than a few non-empty folders without an error and the the operation breaking.

Again, for the first several months of the current setup, using it daily, it’s always worked like a dream. Then suddenly, I had a few hiccups and we’re at the current state.

In your last post, you actually highlighted something I mentioned in my original post. Which was the concept of there being a cache somewhere that’s gotten corrupted and hasn’t refreshed/cleared itself. I tried doing a bit of research on the idea, but didn’t really find any leads. But, given the number of fixes I’ve tried, I would think that would have happened weeks ago. I’ve recreated and remounted the share well over a dozen times, renamed folders and etc. And again, accessing from any other devices, Mac, Windows PC, raspberry Pi OS, all work fine. Even bash navigation works fine. There’s something specific with Dolphin that’s the source.

I prefer dolphin’s interface and feature set for daily use. It’s not really the source of the problem, perse, something else therein has gone awry. I know there’s more powerful file managers than dolphin. I have krusader installed, in fact. But, it’s a bit too much for my work flow on the daily.

But again, thanks for your advice thus-far. It’s at-least helped some, and got me looking at different methods and concepts.

10

There has been a minor bug-fix release relating to smbclient just yesterday on unstable branch.

From v.419.4 → 4.19.5

https://packages.manjaro.org/?query=smbclient

11

UPDATE!

I appreciate all that have posted suggestions. However, as of a few days ago, the problem has resolved itself!

I don’t follow the Manjaro release schedule, I take the releases as they come, and apply them like wise. However, a few days go I noticed there was a large release, with a total package download to be around 1.5GB. I was actually hoping for that to happen, thinking that might actually help my issue. Well, it did.

I applied the latest release updates, restarted the VM, and it literally fixed everything! Responsive and instantaneous file listings while navigating, no missing files and no errors when copying files to the network share!

Now, bear in mind, we still don’t know what the actual culprit was. It could have been a previous release that introduced a bug of some sort, which this current release straightened out. Or, something on my specific system may have gotten corrupted, for whatever reason, and the process of updating to the latest stable release inadvertently straightened out the issue I was having. Regardless, it seems be be resolved now!

I’m going to give it a couple more days, and then mark this thread as solved.

Thanks again!

12

That is the schedule.

13

What one prefer an what is working is not always the same …

Linux is not Windows - and therefore a lot of things must be done differently.

If you prefer the ‘Windows’ way - use Windows - otherwise learning how to do it on Manjaro is you best path forward.

14

This topic was automatically closed 36 hours after the last reply. New replies are no longer allowed.