Slow USB transfers

Even this is no scientific test - the results speaks for themselves.

You can draw your own conclusions - to me it is evident the LInux kernel is not the culprit …

 $ inxi -SC
System:
  Host: tiger Kernel: 6.5.0-rc7-next-20230824-1-next-git-12477-g2b3bd393093b
    arch: x86_64 bits: 64 Desktop: KDE Plasma v: 5.27.7 Distro: Manjaro Linux
CPU:
  Info: 12-core model: AMD Ryzen Threadripper PRO 5945WX s bits: 64
    type: MT MCP cache: L2: 6 MiB
  Speed (MHz): avg: 2269 min/max: 1800/7015 cores: 1: 1800 2: 1800 3: 1800
    4: 1800 5: 1800 6: 2940 7: 1739 8: 1800 9: 1800 10: 4100 11: 1800 12: 2940
    13: 1800 14: 1740 15: 1800 16: 1800 17: 1800 18: 4100 19: 1800 20: 1756
    21: 1757 22: 4100 23: 1800 24: 4100

Depending on your system and on the source - e.g. another disk device - reproducing will be different as these tests does not read another disk device but a stream of random bytes produced by the system. The reason for using thr random stream is simply to avoid identical streams when testing the various devices.

Reproduce the tests

Ensure your system is fully up-to-date and hdparm installed

sudo pacman -Syu hdparm

Create this udev rule (referenced earlier)

 $ cat /etc/udev/rules.d/99-usb-sync.rules 
# rule to disable write cache for usb storage
# requires hdparm to be installed
ACTION=="add|change", KERNEL=="sd[a-z]", ENV{ID_USB_TYPE}=="disk", RUN+="/usr/bin/hdparm -W 0 /dev/%k"

#
# the following rules is introduced with kernel 6.2
# https://docs.kernel.org/admin-guide/abi-testing.html#abi-sys-class-bdi-bdi-strict-limit
# https://docs.kernel.org/admin-guide/abi-testing.html#abi-sys-class-bdi-bdi-max-ratio
# https://docs.kernel.org/admin-guide/abi-testing.html#abi-sys-class-bdi-bdi-max-bytes
ACTION=="add|change", KERNEL=="sd[a-z]", ENV{ID_USB_TYPE}=="disk", RUN+="/usr/bin/echo 1 > /sys/block/%k/bdi/strict_limit",  RUN+="/usr/bin/echo 50 > /sys/block/%k/bdi/max_ratio", RUN+="/usr/bin/echo 16777216 > /sys/block/%k/bdi/max_bytes"

Reload the rules

sudo udevadm control --reload

Create a test script - requires the package time to be installed.

 $ cat test.sh
#!/usr/bin/env bash
echo "Executing: time dd if=/dev/urandom of=/mnt/test.img bs=1G count=100 status=progress"
time sudo dd if=/dev/urandom of=/mnt/test.img bs=1G count=100 status=progress
echo "Executing: time umount /mnt"
time sudo umount /mnt

You can create variations over this to simulate other conditions e.g.

# create 100 files of 100M each
for i in {1..100}; do
    sudo dd if=/dev/urandom of="/mnt/test${i}.img" bs=10M count=10
done

Mount the test device/partition on /mnt and execute …

sudo mount /dev/sdxY /mnt

SATA enclosure test

OCZ Agility 3 240G SATA in 1.gen USB 3 enclosure (? 2010)

formatted with ext4

 $ sudo bash test.sh
Executing: time dd if=/dev/urandom of=/mnt/test.img bs=1G count=100 status=progress
107374182400 bytes (107 GB, 100 GiB) copied, 664 s, 162 MB/s
100+0 records in
100+0 records out
107374182400 bytes (107 GB, 100 GiB) copied, 735,558 s, 146 MB/s

real    12m15,586s
user    0m0,005s
sys     0m0,006s
Executing: time umount /mnt

real    0m9,505s
user    0m0,004s
sys     0m0,008s

formatted with ntfs

 $ sudo bash test.sh
Executing: time dd if=/dev/urandom of=/mnt/test.img bs=1G count=100 status=progress
107374182400 bytes (107 GB, 100 GiB) copied, 676 s, 159 MB/s
100+0 records in
100+0 records out
107374182400 bytes (107 GB, 100 GiB) copied, 675,717 s, 159 MB/s

real    11m15,742s
user    0m0,011s
sys     0m0,000s
Executing: time umount /mnt

real    0m59,460s
user    0m0,004s
sys     0m0,007s

USB-C device test

Samsung 1.8T T7 Shield USB-C (july 2023)

formatted with ntfs

 $ sudo bash test.sh
Executing: time dd if=/dev/urandom of=/mnt/test.img bs=1G count=100 status=progress
107374182400 bytes (107 GB, 100 GiB) copied, 485 s, 221 MB/s
100+0 records in
100+0 records out
107374182400 bytes (107 GB, 100 GiB) copied, 485,176 s, 221 MB/s

real    8m5,205s
user    0m0,004s
sys     0m0,007s
Executing: time umount /mnt

real    0m15,391s
user    0m0,002s
sys     0m0,007s

This device proved a 3-4 minute improvement in transfer speed compared to the old SATA device

WD Passport 512G MILAN II USB-C (? 2021)

formatted with ext4

 $ subo bash test.sh
Executing: time dd if=/dev/urandom of=/mnt/test.img bs=1G count=100 status=progress
107374182400 bytes (107 GB, 100 GiB) copied, 619 s, 174 MB/s
100+0 records in
100+0 records out
107374182400 bytes (107 GB, 100 GiB) copied, 685,141 s, 157 MB/s

real    11m25,165s
user    0m0,012s
sys     0m0,000s
Executing: time umount /mnt

real    0m6,726s
user    0m0,010s
sys     0m0,000s

64G USB stick test

SanDisk Extreme 64G USB 3.0 (? 2020)

This is tested like the others but using a 50G stream

 $ cat test-50.sh
#!/usr/bin/env bash
echo "Executing: time dd if=/dev/urandom of=/mnt/test.img bs=1G count=100 status=progress"
time sudo dd if=/dev/urandom of=/mnt/test.img bs=1G count=100 status=progress
echo "Executing: time umount /mnt"
time sudo umount /mnt

formatted with exfat

 $ sudo bash test-50.sh
Executing: time dd if=/dev/urandom of=/mnt/test.img bs=1G count=50 status=progress
53687091200 bytes (54 GB, 50 GiB) copied, 869 s, 61,8 MB/s
50+0 records in
50+0 records out
53687091200 bytes (54 GB, 50 GiB) copied, 868,694 s, 61,8 MB/s

real    14m28,720s
user    0m0,004s
sys     0m0,004s
Executing: time umount /mnt

real    2m19,978s
user    0m0,001s
sys     0m0,005s

formatted with ntfs

 $ sudo bash test-50.sh
Executing: time dd if=/dev/urandom of=/mnt/test.img bs=1G count=50 status=progress
53687091200 bytes (54 GB, 50 GiB) copied, 740 s, 72,5 MB/s
50+0 records in
50+0 records out
53687091200 bytes (54 GB, 50 GiB) copied, 740,327 s, 72,5 MB/s

real    12m20,350s
user    0m0,002s
sys     0m0,006s
Executing: time umount /mnt

real    3m11,227s
user    0m0,006s
sys     0m0,000s

formatted using f2fs

 $ sudo bash test-50.sh
Executing: time dd if=/dev/urandom of=/mnt/test.img bs=1G count=50 status=progress
53687091200 bytes (54 GB, 50 GiB) copied, 701 s, 76,6 MB/s
50+0 records in
50+0 records out
53687091200 bytes (54 GB, 50 GiB) copied, 700,81 s, 76,6 MB/s

real    11m40,822s
user    0m0,001s
sys     0m0,008s
Executing: time umount /mnt

real    2m35,335s
user    0m0,000s
sys     0m0,006s

Kingston DataTraveler100 64G USB 3.1 (? 2021)

This device had been tested to showcase vendor quality

formatted using ntfs

 $ sudo bash test-50.sh
Executing: time dd if=/dev/urandom of=/mnt/test.img bs=1G count=50 status=progress
53687091200 bytes (54 GB, 50 GiB) copied, 1104 s, 48,6 MB/s
50+0 records in
50+0 records out
53687091200 bytes (54 GB, 50 GiB) copied, 1103,54 s, 48,6 MB/s

real    18m25,257s
user    0m0,008s
sys     0m0,000s
Executing: time umount /mnt

real    6m24,581s
user    0m0,003s
sys     0m0,003s