KIO-Fuse sftp : Files fill memory buffer/cache before being used

Support KDE Plasma
1

Hello,
Since my last updates, I’ve felt that my videos files opened on my sftp network shares via KIO-Fuse take too much time to open. I also realized that the bigger my files were, the longer it took.
I am a heavy user of network shares (sftp , smb ) and KIO-Fuse was the thing that make me definitely switch to KDE.
After investigation, I discovered that when i try to play (use) a file from sftp share via KIO-FUSE, it is fully copied in my memory cache/buffer before getting used by the application (ex : mpv or vlc). And it only happens for sftp share
For example :
I want to play a 5GB mp4 file with mpv :

KIO-FUSE Sftp : I found a procces call sftp.so that is “downloading” my file , my memory consumption raise with the 5GB and then mpv start and play my video.

SSHFS mount : mpv start directly , my file is “streamed” and my memory consumption don’t grow.

KIO-FUSE SMB : mpv start directly , my file is “streamed” and my memory consumption don’t grow.

Something weird, I’ve only examples with videos files but file copying works normally (via Midnight Commander for example)

I think something change somewhere with Kio , Kio fuse , ssh. But i can’t figure it out where it is

PS : Sorry for my bad English (french guy here)

2

:+1: Welcome to Manjaro! :+1:

Yep! This is by design and why KDE works and none of the other DEs will work as you have to download the file manually before playing it.

Pas mal ton :uk: ! :grin:

3

Hi , thanks for your answer.
But the behavior as i described only happening recently , i don’t really know since which update but not much as few weeks :
Before : (lets say KDE 5.21)
Play my video from my sftp (kio-fuse’d) share (via dolphin , terminal , anything) with mpv , vlc . Player launch immedialty and video play.

Now :
Something retrieve my video in background , fill the memory buffer/cache with it (i have 16GB Ram it ok for most files but it’s not it supposed to work i think ) and then when the file is “memory baked” . My video player launch and play the file (depends of my network , but it can be several minutes later)

It only happens on SFTP share , SMB share works as before

4

It’s always been like that since KDE Plasma (5.0) and if you have a slow network connection on a samba share, slow DVD drive, … it downloads it as well.

And it’s not using up any free RAM, but it’s using your buffer/cache space:

free --human
               total        used        free      shared  buff/cache   available
Mem:            15Gi       6.4Gi       474Mi       154Mi       8.7Gi       8.7Gi
Swap:           19Gi       588Mi        19Gi

and if you don’t have enough buffers/cache to hold the file, it’ll write the file in ~/.cache/kioexec

:thinking:

5

Hello,
I’m sorry to insist , but it’s not always have been like that. I made 2 videos samples

Same file , same place. I take a 20GB video to be sure it’s not fit in my memory.
First with sftp share, you can see mpv launch and wait for the video, my memory grow , sftp.so process downloading the file in memory , then something KILL the process surely cause it’s taking to much memory. (it work with smaller file that fit in memory but i have to wait until the file is completely memory baked)
Second , same file with samba share , mpv launch , and the video play directly , memory don’t grow.

And i insist , sftp used to work like the second video (smb), it only happens recently

I go find the videos there

gofile .io/d/0c1g6A

I know that before KIO-FUSE , i had to download file from network share before use them with non-KDE apps (mpv,vlc,…) , thats why i used to be on GVFS compatible DE (Cinnamon, Xfce) , but since KIO-FUSE it work like charm … until now.

1 Like
6