Cloning a Disk Quickly?

I'm looking to clone one of my computers to an SSH server .iso file.

I'm currently using dd, but it's transferring over SSH at about 30MB/s and is taking forever.

I saw a method using cat, but am not sure how to do this over ssh.

What is a fast way to clone a disk to a remote SSH besides using clonezilla?

What is the network between the two machines and what type of disk are you writing to on the other side? 30MB/s is decently fast over ssh. That is about ~240Mbps.

Wow, really?

The connection is ethernet-ethernet and it's wired to the same network switch.

So you just think I'm being impatient?

I don't know, I read that dd is rather slow compared to cat or other tools.

I am not sure if I understand what you want to achieve? dd is copying each block of the device - even empty blocks. This makes it slow by definition.

Do you really need a 1:1 clone including all empty blocks of the device?

Also, it would be worthwhile to see the full command you are using: ssh + dd
Which parameters do you use?
Filesystem target/source?
Blocksize target/source?

Regarding network speed:
10 MB/s corresponds to a 100 MBit LAN
100 MB/s corresponds to a 1 GBit LAN

CloneZilla allows its cloning process to be used with ssh and knows about sparse files, compression, ...

I've never used it that way and you will have to experiment in balancing compression vs network speed (which is faster: your CPU or the network?) to get the optimal speed, but once you get it nailed down, there is nothing faster...

The only other way except dd and CloneZilla I know about is ddrescue but that's probably not a use case you're looking for... (just mentioning it for completeness)

I think so.

My mom's old computer bit the dust and she wants to port the entire system (The exact way she left it) to the new computer.

Here is the command I used: ssh SciLab0 -C "dd if=/home/sarah/Mama_HDD/Kubuntu_HDD.iso bs=1M" | sudo dd of=/dev/sda bs=1M status=progress

This is achieving about 56MB/s according to the output.

You can maybe compress the backup before sending it over a bottleneck.
https://wiki.archlinux.org/index.php/Full_system_backup_with_SquashFS
https://wiki.archlinux.org/index.php/Full_system_backup_with_tar
Use a compression which gives you the best combination of size and speed. LZ4 is best for speed, ZSTD can be tweaked to give a good combination of speed and size.

1 Like

I am pretty sure she is not interested in empty blocks. :sweat_smile:

Your speed is pretty decent. 56 MB/s is not bad. I would check how much the old HD is able to deliver. What is the speed if you do a local dd copy from one folder to the other? That should be your benchmark.

I am not a dd expert but I am wondering about bs=1M. Shouldnt that be the blocksize of the underlying device?

Not today any more I would even say to go as high as 16M if doing disk-to-disk...

Over a network it depends on a lot of parameters: OS, Kernel version, latency, TCP window size, ... I have a calculator somewhere that takes this into account.

What about the read/write speeds of both hard drives? Lot of people overlook that and think that because they have a good network that it will be quick. You're at the mercy of the slowest hard drive, or your network if it transmits slower than the hard drives can read/write.

1 Like

Forum kindly sponsored by