TL;DR version:
Does Dolphin support transfer queue? If so, how to enable it?
Long version:
Every now and then, I need to perform data migration in the quantity of GB and TB.
I use Dolphin for the data migration.
Scenario A:
If the migration involves HDD 1 to HDD 2, AND HDD 3 to HDD 4, of course the 2 transfer can happen in parallel without impact to the transfer speed.
Scenario B:
If the migration involves HDD 1 to HDD 2, AND HDD 1 to HDD 3, then it is not ideal to run both transfers simultaneously.
In Scenario B, I guess the solution would be to perform the transfer in sequential, ie. initiate 2nd transfer after the 1st has completed.
However, as the transfer involves GB or TB of data, I wouldn’t be able to wait by the desktop to initiate 2nd transfer.
And it would be ideal to let the system initiate 2nd transfer automatically, to save time.
So, is there a way to do it in Dolphin?
If so, how to do it?
Thank you!
PS: I need to retain the file properties of each file. And those sync apps seem to reset the date/time properties of the files to the date/time of transfer. So I cannot use sync app.
Dolphin Version 24.08.3
Operating System: Manjaro Linux
KDE Plasma Version: 6.2.4
KDE Frameworks Version: 6.8.0
Qt Version: 6.8.1
Kernel Version: 6.12.1-4-MANJARO (64-bit)
I wouldn’t think such a feature would be built in - however, you can write commands and scripts to do what you need to do.
I’m not sure I’d do both simultaneously, especially as we’re talking about Hard Disks here and not SSD’s.
Now, for transferring a LARGE amount of data - I’d be looking at rsync, which is safer and more reliable.
Archiving preserves files status…
alias rsync-progress='rsync -avh --progress'
alias rsync-stat='rsync --progress --stats -ravz'
Look at man rsync to decypher those
Now how do you check?
command_that_might_fail if [ $? -ne 0 ]; then echo "The command failed." else echo "The command succeeded."
Obviously, Dolphin also supports (F4) Konsole, so you could open a couple of dual pane windows - but I’d suggest triggering them individually if you’re going to do it that way, or simply write the script:
The Script
hidden
#!/bin/bash
# Function to check the status of the last command and exit if an error occurred
check_status() {
if [ $? -ne 0 ]; then
echo "An error occurred. Exiting script."
exit 1
fi
}
# Move data from /mnt/hd1 to /mnt/hd2
echo "Starting transfer from /mnt/hd1 to /mnt/hd2..."
rsync -avh --progress /mnt/hd1/ /mnt/hd2/
check_status
echo "Transfer from /mnt/hd1 to /mnt/hd2 completed successfully."
# Move data from /mnt/hd3 to /mnt/hd4
echo "Starting transfer from /mnt/hd3 to /mnt/hd4..."
rsync -avh --progress /mnt/hd3/ /mnt/hd4/
check_status
echo "Transfer from /mnt/hd3 to /mnt/hd4 completed successfully."
echo "All transfers completed successfully."
Don’t trust me, make sure you test this out properly (because I didn’t, and because I’m not a Guru)…
Lack of experience would not let me enter into a discussion of sequential or parallel transfers.
As long as the 2 transfer sessions involve different HDDs, the transfer bandwidth won’t be throttled.
And due to the fact that these are HDDs and not SSDs, parallel transfer would save 50% of time, compared to sequential transfer.
Yeah, I learnt my lesson - especially my area suffers from unexpected power outage.
The trick is: I copy the files, and upon completion, run a file comparison on both HDDs, then only I delete the files on source HDD.
If there is any power outage or system hang, I can backtrack the file transfer progress and resume transfer, with a little effort.
Hmm, I didn’t know rsync has archive mode, which preserve the file properties.
I will look up to see how to use this, or find a GUI version of it.
Thanks!
Deleting the original files only if the copy was successful.
Running both operations in parallel and includes checks to ensure data integrity:
#!/bin/bash
# Function to check the status of the last command and exit if an error occurred
check_status() {
if [ $? -ne 0 ]; then
echo "An error occurred. Exiting script."
exit 1
fi
}
# Start first transfer
echo "Starting transfer from /hdd1 to /hdd2..."
rsync -avh --progress --remove-source-files /hdd1/ /hdd2/ > /tmp/rsync_hd1_hd2.log 2>&1 &
PID1=$!
# Start second transfer in the background and log the output
echo "Starting transfer from /hdd3 to /hdd4..."
rsync -avh --progress --remove-source-files /hdd3/ /hdd4/ > /tmp/rsync_hd3_hd4.log 2>&1 &
PID2=$!
# Wait for both background jobs to finish
wait $PID1
check_status
echo "Transfer from /hdd1 to /hdd2 completed successfully."
wait $PID2
check_status
echo "Transfer from /hdd3 to /hdd4 completed successfully."
echo "All transfers completed successfully."
Well, you might be out of luck then. Unless maybe spend some time to familiarise yourself with Krusader, and use that when you want to use the file queue feature.
And there’s always UltraCopier from the AUR (mentioned before) but that will be without support from Manjaro; plus it’s outdated; but, maybe it might surprise you.
Another idea.
How about using grsync at the same time?
I use grsync for large-scale moves.
However, unlike when using nemo, it does not copy user-defined emblems etc. I feel like there must be a better way, but I have never tried it successfully.
If there are 10,000 target files and only one or two are corrupted, nemo is very inconvenient.
For this, I have a quick solution: FreeFileSync.
It is a GUI app.
It can quickly scan and compare multiple files and show u the different folders/files in 2 sources, then u can decide which direction to sync.
Don’t trust me, make sure you test this out properly (because I didn’t, and because I’m not a Guru)…
(air guru, maybe)
