Slow USB transfers

What you see could be the effect of the kernel cache.

If you want to draw any conclusions on a sudden drop it could be the stick’s internal software doing error correction, which then point to possible defective memory cells.

What you should look at is the final numbers.

The amount of time taken to write the data and the amount of time taken to unmount the device - which would include flush what remains of the buffer.

What happens inbetween is skewed by cache is not a reliable number as it will be different running the same test twice.

Please, try to focus on solving your specific issue and avoid negative general assessments which could be deemed as rant against Linux.

1 Like

I tried to rsync my data from my internal Nvme with Ext4 to my external USB stick 64 GB with Exfat via USB 3.0.

I created a shell script.

You need to replace <SOURCE> with your file path, <TARGET> too.


echo "===Start==="
echo "RAM cache:"

for i in {1..8}; do
    printf "\n===%s.repeat===\n" "$i"
    echo "Rsync your data:"
    rsync --progress "<SOURCE>" "<TARGET>/$i"

echo ""
echo "===End==="
echo "RAM cache:"


RAM cache:
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 24244052  61712 2351780    0    0 17946  1531 10831    7  2  2 95  1  0

Rsync your data:
  2.108.896.285 100%  518,21MB/s    0:00:03 (xfr#1, to-chk=0/1)

Rsync your data:
  2.108.896.285 100%  675,34MB/s    0:00:02 (xfr#1, to-chk=0/1)

Rsync your data:
  2.108.896.285 100%  138,89MB/s    0:00:14 (xfr#1, to-chk=0/1)

Rsync your data:
  2.108.896.285 100%   55,68MB/s    0:00:36 (xfr#1, to-chk=0/1)

Rsync your data:
  2.108.896.285 100%   57,13MB/s    0:00:35 (xfr#1, to-chk=0/1)

Rsync your data:
  2.108.896.285 100%   24,94MB/s    0:01:20 (xfr#1, to-chk=0/1)

Rsync your data:
  2.108.896.285 100%   24,53MB/s    0:01:22 (xfr#1, to-chk=0/1)

Rsync your data:
  2.108.896.285 100%   26,23MB/s    0:01:16 (xfr#1, to-chk=0/1)

RAM cache:
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 2  1    256 430224  36460 25875708    0    0  7662 34079 4241    3  1  1 88 10  0

RAM showed small free space after the transfer.
I know now that the transfer speed depends on the temperature of my USB stick.
When temperature is higher, speed will be reduced more dramatically.

I would suggest a solution. The USB stick needs a pause of 1 min due to overheating after transferring data 10 GB. After the pause then it goes faster.

Edit:// The cache transfer was used in my test, which is a trick. My test is unfair.

1 Like

i already pointed out that a lot of usb-storages really suffer due to thermal problems, the resulting transfer-errors and the additional error-correction that must be done with more syncing, but the TO declined. there is one pro-argument of the TO that he reports no issues with MS-Windows.
there is in fact a different between the handling of large file-transfers between windows and linux.

this is a interesting thread. i was googeling for a long time. can it be that one main problem is the synced and cached transfer to the usb? i’ve got no usb-mass-media for testing this but if anyone can test to mount the usb-device in async mode and flush to a folder, copy a large amount of data inside this and check if the speed doesn’t collapse ?

mount -o async,flush -t ntfs /dev/sdb1 ~/somewhere

(fuse-exfat package is needed)

You are right after my tests. The cached transfer is a trick.

sudo mount -t exfat -o async,uid=1000,gid=1000 /dev/sda1 /mnt/

With cache:

RAM cache:
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 2  0      0 18555080  86648 6537648    0    0  3640  6544 4136    3  1  1 97  2  0

Rsync your data:
  2.108.896.285 100%  674,21MB/s    0:00:02 (xfr#1, to-chk=0/1)

RAM cache:
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 16010948  86648 9024236    0    0  3632  6530 4149    3  1  1 97  2  0

I disabled cache:

echo 16777216 > /sys/block/sda/bdi/max_bytes
echo 1 > /sys/block/sda/bdi/strict_limit

RAM cache:
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 18078912  94088 6702120    0    0  2958  6543 4306    3  1  1 97  1  0

Rsync your data:
  2.108.896.285 100%   37,40MB/s    0:00:53 (xfr#1, to-chk=0/1)

RAM cache:
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 15544208  94204 9188888    0    0  2870  7507 4267    3  1  1 97  2  0
1 Like

well i found an old usb-storage-device with ntfs on it and did the following:
first i mounted the device manually in async mode as below and copied for ~800 seconds
to it. then i unmounted the device, mounted it via the ordinary mount option of the system with standardsettings as removable/external usb-device and copied another ~800 seconds
a new file to it. the difference is indeed significant, it’s 6,9 GB against 4,8 GB in standard-mode.

note: this is an old usb-storage device with no modern caching-electronics inside. therefore the difference must be the simple fact of syncing.

   ~  sudo mount -o async,uid=1000,gid=1000 -t ntfs /dev/sdg1 /home/OLLI/TEST                                                                                             

   ~  dd if=/dev/urandom of=/home/OLLI/TEST/TEST/test_004.txt status=progress                                                                                              

6876852736 Bytes (6,9 GB, 6,4 GiB) kopiert, 805 s, 8,5 MB/s^C
13435822+0 Datensätze ein
13435821+0 Datensätze aus
6879140352 Bytes (6,9 GB, 6,4 GiB) kopiert, 805,708 s, 8,5 MB/s

    ~  sudo umount /dev/sdg1                                                                                                            
    ~  dd if=/dev/urandom of=/run/media/OLLI/SeagateSlimDrive/TEST/test_005.txt status=progress                                                                     
4834000896 Bytes (4,8 GB, 4,5 GiB) kopiert, 804 s, 6,0 MB/s^C
9446147+0 Datensätze ein
9446147+0 Datensätze aus
4836427264 Bytes (4,8 GB, 4,5 GiB) kopiert, 804,864 s, 6,0 MB/s

A second look let me think that it writes nothing yet. Everything goes to RAM here, since:

Actual DISK WRITE:       0.00 B/s

When I observe 0 here, then I see also no activity on my USB HDD.

That let me question if here is problem with the RAM? Why is the transfer to RAM so slow? Fragmentation of NTFS/exFAT?

Maybe try:

rsync -av --progress --partial --fsync /source/ /target/source

So with --fsync. That will start the next file after is was really synced/written to the disk.

i was testing a little bit more. the TO reported that the changes of the vm_dirty settings had no effect. there is a complete difference to my tests. i redo the test that i already posted, same hardware, no change to the hardware.
i edited a



the testresult is stunning. first the test by mounting the drive manually with asynchronus mode:

sudo mount -o async,uid=1000,gid=1000 -t ntfs /dev/sdg1 /home/OLLI/TEST

dd if=/dev/urandom of=/home/OLLI/TEST/TEST/test_001.txt status=progress                                                                                              ✔

7391458816 Bytes (7,4 GB, 6,9 GiB) kopiert, 805 s, 9,2 MB/s^C
14439745+0 Datensätze ein
14439745+0 Datensätze aus
7393149440 Bytes (7,4 GB, 6,9 GiB) kopiert, 805,12 s, 9,2 MB/s

the comparision-results:
first test: 6,9 GB @ 8,5 MB/s
second test (vm_dirty changed): 7,4 GB 9,2 MB/s
that’s another +7% change

but now it gets impressive, the second test with standard-settings of the system mounting

dd if=/dev/urandom of=/run/media/OLLI/SeagateSlimDrive/TEST/test_002.txt status=progress

8593174528 Bytes (8,6 GB, 8,0 GiB) kopiert, 804 s, 10,7 MB/s^C
16784703+0 Datensätze ein
16784703+0 Datensätze aus
8593767936 Bytes (8,6 GB, 8,0 GiB) kopiert, 804,579 s, 10,7 MB/s

first test: 4,8 GB @ 6,0 MB/s
second test (vm_dirty changed): 8,6 GB @ 10,7 MB/s
that’s a +79% !

now the synced transfer is even faster than the asynchronus file transfer.
changing the vm_dirty settings has a huge impact.

I havent rechecked the values, etc on this proof-of-concept in a little while
(from back when garuda first started lobbing these configs with a hard value on all systems)

1 Like

:+1: :ok_hand: :point_up:

using the shell-script and accepted the given recommended values are perfect.
redoing the test, there was no drop of file-transfer-bandwith while copying and the result:

dd if=/dev/urandom of=/run/media/OLLI/SeagateSlimDrive/TEST/test_002.txt status=progress                                                                            

11972704768 Bytes (12 GB, 11 GiB) kopiert, 807 s, 14,8 MB/s^C
23410600+0 Datensätze ein
23410600+0 Datensätze aus
11986227200 Bytes (12 GB, 11 GiB) kopiert, 807,903 s, 14,8 MB/s

12 GB @ 15 MB/s

but now it’s getting weird. i started copying multiple files at once to the drive

dd if=/dev/urandom of=/run/media/OLLI/SeagateSlimDrive/TEST/test_003.txt status=progress                                                                            
4959769088 Bytes (5,0 GB, 4,6 GiB) kopiert, 479 s, 10,4 MB/s^C

dd if=/dev/urandom of=/run/media/OLLI/SeagateSlimDrive/TEST/test_004.txt status=progress               
1551412224 Bytes (1,6 GB, 1,4 GiB) kopiert, 195 s, 8,0 MB/s^C

dd if=/dev/urandom of=/run/media/OLLI/SeagateSlimDrive/TEST/test_005.txt status=progress                
1243583488 Bytes (1,2 GB, 1,2 GiB) kopiert, 168 s, 7,4 MB/s^C

dd if=/dev/urandom of=/run/media/OLLI/SeagateSlimDrive/TEST/test_006.txt status=progress               
213726720 Bytes (214 MB, 204 MiB) kopiert, 36 s, 5,9 MB/s^C

that brings up a bandwith of 31,7 MB/s while the bandwith of a single file is limited to ~15MB/s and does not exceed this. is there anything that limits a single file copy to the half of the max-bandwith ?

Hm. I couldnt say for sure - thats for you grubby testers to do :stuck_out_tongue_winking_eye:

Of course my first guess is just good utilization of cache - you cant achieve quite as much speed as a single operation but multiple times at once (not quite as fast, but greater than the simple sum of single speed divided by number of operations).

well from what i found out till now:
this is a USB-2 device. the bandwith is 480MBit/s,therefor the theoretical bandwith would be about 50 MByte/s BUT every USB-connector at the PC is shared with some others (2-4 connections share one controller). this and the overhead of the usb-protocol lead to a bandwith of ~30-36 MBytes/s @USB2. all in the drive shows up that it is correct working and produces the output of 32 MBytes/s. the only thing i can’t understand till now is that the bandwith at a single-file copy is limited to ~ 1/2 of the max speed. but it is constant and i can reproduce it 100% at my system.
i also tried to drip the last bytes out with ionice (used c1 c2 c3…) but there was just a few more output.

ionice -c 1 dd if=/dev/urandom of=/run/media/OLLI/SeagateSlimDrive/TEST/test_103.txt status=progress 

Is this a serious question? Well it can limit them, thats for sure.

Why is it that everyone misses the fact that it works fine under windows and works fine using dd on linux(though a bit slower)? Is windows and dd magically influencing the hardware? :slight_smile: Are we supposed to just live like this because we are used to it? 2 hours vs almost 2 days is big difference when transferring files.

Here you go:

inxi --admin --usb --cpu --machine --disk --partitions --swap                                        ✔ 
  Type: Desktop System: Dell product: OptiPlex 3020 v: 01
    serial: <superuser required> Chassis: type: 6 serial: <superuser required>
  Mobo: Dell model: 0VHWTR v: A02 serial: <superuser required> UEFI: Dell
    v: A02 date: 01/07/2014
  Info: model: Intel Core i3-4130 bits: 64 type: MT MCP arch: Haswell
    gen: core 4 level: v3 note: check built: 2013-15 process: Intel 22nm
    family: 6 model-id: 0x3C (60) stepping: 3 microcode: 0x28
  Topology: cpus: 1x cores: 2 tpc: 2 threads: 4 smt: enabled cache:
    L1: 128 KiB desc: d-2x32 KiB; i-2x32 KiB L2: 512 KiB desc: 2x256 KiB
    L3: 3 MiB desc: 1x3 MiB
  Speed (MHz): avg: 3394 high: 3400 min/max: 800/3400 scaling:
    driver: intel_cpufreq governor: schedutil cores: 1: 3393 2: 3392 3: 3400
    4: 3392 bogomips: 27149
  Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx
  Type: itlb_multihit status: KVM: VMX disabled
  Type: l1tf mitigation: PTE Inversion; VMX: conditional cache flushes, SMT
  Type: mds mitigation: Clear CPU buffers; SMT vulnerable
  Type: meltdown mitigation: PTI
  Type: mmio_stale_data status: Unknown: No mitigations
  Type: retbleed status: Not affected
  Type: spec_store_bypass mitigation: Speculative Store Bypass disabled via
  Type: spectre_v1 mitigation: usercopy/swapgs barriers and __user pointer
  Type: spectre_v2 mitigation: Retpolines, IBPB: conditional, IBRS_FW,
    STIBP: conditional, RSB filling, PBRSB-eIBRS: Not affected
  Type: srbds mitigation: Microcode
  Type: tsx_async_abort status: Not affected
  Local Storage: total: 2.79 TiB used: 305.77 GiB (10.7%)
  SMART Message: Unable to run smartctl. Root privileges required.
  ID-1: /dev/sda maj-min: 8:0 vendor: Seagate model: ST2000DM006-2DM164
    size: 1.82 TiB block-size: physical: 4096 B logical: 512 B speed: 6.0 Gb/s
    tech: HDD rpm: 7200 serial: Z4Z7P1TM fw-rev: CC26 scheme: MBR
  ID-2: /dev/sdb maj-min: 8:16 vendor: Seagate model: ST500DM002-1BD142
    size: 465.76 GiB block-size: physical: 4096 B logical: 512 B speed: 6.0 Gb/s
    tech: HDD rpm: 7200 serial: Z6E3J1HP fw-rev: KC48 scheme: GPT
  ID-3: /dev/sdc maj-min: 8:32 vendor: Seagate model: ST3320820AS
    size: 298.09 GiB block-size: physical: 512 B logical: 512 B speed: 3.0 Gb/s
    tech: N/A serial: 9QF599RP fw-rev: G scheme: MBR
  ID-4: /dev/sdd maj-min: 8:48 model: USB Disk 3.0 size: 116.21 GiB
    block-size: physical: 512 B logical: 512 B type: USB rev: 3.2 spd: 5 Gb/s
    lanes: 1 mode: 3.2 gen-1x1 tech: N/A serial: FC23356D74ED5052 fw-rev: 2.00
    scheme: MBR
  SMART Message: Unknown USB bridge. Flash drive/Unsupported enclosure?
  ID-5: /dev/sde maj-min: 8:64 model: USB Disk 3.0 size: 116.21 GiB
    block-size: physical: 512 B logical: 512 B type: USB rev: 2.1 spd: 480 Mb/s
    lanes: 1 mode: 2.0 tech: N/A serial: FC211729179556F6 fw-rev: 3.00
    scheme: MBR
  SMART Message: Unknown USB bridge. Flash drive/Unsupported enclosure?
  Message: No partition data found.
  **Alert: No swap data was found.**
  Hub-1: 1-0:1 info: full speed or root hub ports: 2 rev: 2.0
    speed: 480 Mb/s (57.2 MiB/s) lanes: 1 mode: 2.0 chip-ID: 1d6b:0002
    class-ID: 0900
  Hub-2: 1-1:2 info: Intel Integrated Rate Matching Hub ports: 4 rev: 2.0
    speed: 480 Mb/s (57.2 MiB/s) lanes: 1 mode: 2.0 chip-ID: 8087:8008
    class-ID: 0900
  Hub-3: 2-0:1 info: full speed or root hub ports: 2 rev: 2.0
    speed: 480 Mb/s (57.2 MiB/s) lanes: 1 mode: 2.0 chip-ID: 1d6b:0002
    class-ID: 0900
  Hub-4: 2-1:2 info: Intel Integrated Rate Matching Hub ports: 6 rev: 2.0
    speed: 480 Mb/s (57.2 MiB/s) lanes: 1 mode: 2.0 chip-ID: 8087:8000
    class-ID: 0900
  Hub-5: 3-0:1 info: hi-speed hub with single TT ports: 10 rev: 2.0
    speed: 480 Mb/s (57.2 MiB/s) lanes: 1 mode: 2.0 chip-ID: 1d6b:0002
    class-ID: 0900
  Device-1: 3-3:3 info: Maxxter Wireless Receiver type: mouse,keyboard
    driver: hid-generic,usbhid interfaces: 2 rev: 1.1 speed: 12 Mb/s (1.4 MiB/s)
    lanes: 1 mode: 1.1 power: 50mA chip-ID: 248a:8363 class-ID: 0301
  Device-2: 3-4:4 info: Usb KeyBoard type: keyboard,HID
    driver: hid-generic,usbhid interfaces: 2 rev: 1.1
    speed: 1.5 Mb/s (183 KiB/s) lanes: 1 mode: 1.0 power: 98mA
    chip-ID: c0f4:01f5 class-ID: 0300
  Device-3: 3-9:5 info: USB Disk 3.0 type: mass storage driver: usb-storage
    interfaces: 1 rev: 2.1 speed: 480 Mb/s (57.2 MiB/s) lanes: 1 mode: 2.0
    power: 100mA chip-ID: 23a5:3379 class-ID: 0806 serial: FC211729179556F6
  Hub-6: 4-0:1 info: super-speed hub ports: 2 rev: 3.0
    speed: 5 Gb/s (596.0 MiB/s) lanes: 1 mode: 3.2 gen-1x1 chip-ID: 1d6b:0003
    class-ID: 0900
  Device-1: 4-1:5 info: USB Disk 3.0 type: mass storage driver: usb-storage
    interfaces: 1 rev: 3.2 speed: 5 Gb/s (596.0 MiB/s) lanes: 1
    mode: 3.2 gen-1x1 power: 144mA chip-ID: 23a5:3379 class-ID: 0806
    serial: FC23356D74ED5052

The flash drive i am now writing to is the last usb device on the list.I am booted using a liveISO, so no swap that I am aware of as the report shows.

I reran the dd test, and on the 3.0 port with it now registering correctly. All other usb devices are on a different port/hub of the motherboard.

still using exFAT filesystem

Executing: time dd if=/dev/urandom of=/mnt/test.img bs=1G count=100 status=progress
107374182400 bytes (107 GB, 100 GiB) copied, 2391 s, 44.9 MB/s
100+0 records in
100+0 records out
107374182400 bytes (107 GB, 100 GiB) copied, 2391.01 s, 44.9 MB/s

This test is 5MB/s faster, but otherwise the same results. I would be okay with this, except for the fact that I get higher speeds under windows, benchmarks above 60MB/s, transfers real files between 25 and 50 with mixed small files, and I get less than 5MB/s on linux when actually transferring anything real yet max 42MB/s on a benchmark.

Rerun of the test using ext4

bash                                                                                         ✔ 
Executing: time dd if=/dev/urandom of=/mnt/test.img bs=1G count=100 status=progress
107374182400 bytes (107 GB, 100 GiB) copied, 4331 s, 24.8 MB/s
100+0 records in
100+0 records out
107374182400 bytes (107 GB, 100 GiB) copied, 4330.76 s, 24.8 MB/s
real    72m10.820s
user    0m0.009s
sys     8m18.656s

this is a full half hour longer, at 1hr 12 min.

I ran the test on the external USB HDD (NTFS), but gave up as it dropped to <5MB/s after a few minutes

I copied that data while it was operating to show the low numbers, they consistency were staying that low, and occasionally hit zero while it was updating on the screen. It wasn’t zero the whole time, just incredibly low… KB to <4MB most of the time. I didn’t realize it said zero when i posted it. This is what it does, either A. it starts out that slow, or B. Goes fast for a while (20 min?) before dropping. I know it is an actual problem, or “issue” lol, because it obviously works fine sometimes for that first 20 minutes. During that initial transfer the speeds are closer to reality. Even when it says “0” beside actual write, the drive is still blinking.

That’s awesome that it worked in your case! But strangely did not in mine. I can hit about 42MB/s consistently with my 3.0 USB flash drive with dd, but not transferring actual data. I have yet to see it transfer any large amount (greater than 80gb, often much less) at proper speeds.

-----Its been a day or two…sorry

I have attached the 128gb 3.0 USB drive to a 3.0 port and booted Windows 7
formatted back to exFAT 128gb drive in 46min.

Transferred 101GB of payload data (from several large folders containing 68465 files or 1.47MB a piece on average) in 66 minutes (the drive stopped blinking immediately, write cache is disabled for this drive under device manager) for a total of 25.4MB/s. This is on par with my earlier tests on windows, except I am transferring smaller files mixed in. The original post about the 262GB was mostly larger contiguous files which should bench higher, and does in fact transfer faster on windows as originally stated. Especially to a USB 3.0 hard drive.

The same USB flash drive formatted a second time and benchmarked using “USB Flash Benchmark” on windows reports this data for the 3.0 flash drive:

Starting benchmark.
Bechmarking G:(FC23356D74ED5052)
Started at 8/27/2023 5:14:59 PM
Filename: G:\test.tmp
Starting large file benchmark. With 100MB file.
16384k:6 Write Speed:26.09MB/s
16384k:6 Write Speed:62.62MB/s
16384k:6 Write Speed:62.43MB/s
16384k:6 Average Write Speed:42.66MB/s
16384k:6 Read Speed:130.26MB/s
16384k:6 Read Speed:130.33MB/s
16384k:6 Read Speed:130.31MB/s
16384k:6 Average Read Speed:130.30MB/s
8192k:12 Write Speed:63.27MB/s
8192k:12 Write Speed:63.01MB/s
8192k:12 Write Speed:62.74MB/s
8192k:12 Average Write Speed:63.01MB/s
8192k:12 Read Speed:130.59MB/s
8192k:12 Read Speed:130.66MB/s
8192k:12 Read Speed:130.82MB/s
8192k:12 Average Read Speed:130.69MB/s
4096k:25 Write Speed:64.29MB/s
4096k:25 Write Speed:66.99MB/s
4096k:25 Write Speed:67.61MB/s
4096k:25 Average Write Speed:66.27MB/s
4096k:25 Read Speed:131.55MB/s
4096k:25 Read Speed:131.66MB/s
4096k:25 Read Speed:131.60MB/s
4096k:25 Average Read Speed:131.60MB/s
2048k:50 Write Speed:67.00MB/s
2048k:50 Write Speed:67.24MB/s
2048k:50 Write Speed:67.70MB/s
2048k:50 Average Write Speed:67.31MB/s
2048k:50 Read Speed:131.93MB/s
2048k:50 Read Speed:132.13MB/s
2048k:50 Read Speed:132.11MB/s
2048k:50 Average Read Speed:132.05MB/s
1024k:100 Write Speed:67.30MB/s
1024k:100 Write Speed:67.18MB/s
1024k:100 Write Speed:67.38MB/s
1024k:100 Average Write Speed:67.29MB/s
1024k:100 Read Speed:132.07MB/s
1024k:100 Read Speed:132.29MB/s
1024k:100 Read Speed:132.28MB/s
1024k:100 Average Read Speed:132.22MB/s
512k:200 Write Speed:67.28MB/s
512k:200 Write Speed:67.61MB/s
512k:200 Write Speed:66.85MB/s
512k:200 Average Write Speed:67.24MB/s
512k:200 Read Speed:132.15MB/s
512k:200 Read Speed:132.44MB/s
512k:200 Read Speed:132.35MB/s
512k:200 Average Read Speed:132.31MB/s
Deleting file.
Starting small file benchmark. With 10MB file.
256k:40 Write Speed:9.18MB/s
256k:40 Write Speed:7.45MB/s
256k:40 Write Speed:8.43MB/s
256k:40 Average Write Speed:8.29MB/s
256k:40 Read Speed:132.22MB/s
256k:40 Read Speed:132.01MB/s
256k:40 Read Speed:132.05MB/s
256k:40 Average Read Speed:132.10MB/s
128k:80 Write Speed:7.45MB/s
128k:80 Write Speed:7.70MB/s
128k:80 Write Speed:7.40MB/s
128k:80 Average Write Speed:7.51MB/s
128k:80 Read Speed:131.85MB/s
128k:80 Read Speed:132.14MB/s
128k:80 Read Speed:131.57MB/s
128k:80 Average Read Speed:131.85MB/s
64k:160 Write Speed:8.71MB/s
64k:160 Write Speed:7.32MB/s
64k:160 Write Speed:8.47MB/s
64k:160 Average Write Speed:8.12MB/s
64k:160 Read Speed:131.80MB/s
64k:160 Read Speed:131.75MB/s
64k:160 Read Speed:131.74MB/s
64k:160 Average Read Speed:131.76MB/s
32k:320 Write Speed:23.96MB/s
32k:320 Write Speed:8.64MB/s
32k:320 Write Speed:7.41MB/s
32k:320 Average Write Speed:10.26MB/s
32k:320 Read Speed:130.97MB/s
32k:320 Read Speed:130.83MB/s
32k:320 Read Speed:130.98MB/s
32k:320 Average Read Speed:130.93MB/s
Deleting file.
Starting tiny file benchmark. With 1MB file.
16k:320 Write Speed:73.62MB/s
16k:320 Write Speed:5.97MB/s
16k:320 Write Speed:44.92MB/s
16k:320 Average Write Speed:14.75MB/s
16k:320 Read Speed:109.53MB/s
16k:320 Read Speed:112.74MB/s
16k:320 Read Speed:110.77MB/s
16k:320 Average Read Speed:111.00MB/s
8k:640 Write Speed:58.96MB/s
8k:640 Write Speed:6.89MB/s
8k:640 Write Speed:38.63MB/s
8k:640 Average Write Speed:15.96MB/s
8k:640 Read Speed:74.79MB/s
8k:640 Read Speed:74.14MB/s
8k:640 Read Speed:72.39MB/s
8k:640 Average Read Speed:73.76MB/s
4k:1280 Write Speed:2.56MB/s
4k:1280 Write Speed:2.57MB/s
4k:1280 Write Speed:2.57MB/s
4k:1280 Average Write Speed:2.57MB/s
4k:1280 Read Speed:10.95MB/s
4k:1280 Read Speed:10.95MB/s
4k:1280 Read Speed:10.92MB/s
4k:1280 Average Read Speed:10.94MB/s
2k:2560 Write Speed:1.89MB/s
2k:2560 Write Speed:1.62MB/s
2k:2560 Write Speed:1.89MB/s
2k:2560 Average Write Speed:1.79MB/s
2k:2560 Read Speed:5.53MB/s
2k:2560 Read Speed:5.54MB/s
2k:2560 Read Speed:5.54MB/s
2k:2560 Average Read Speed:5.53MB/s
1k:5120 Write Speed:1.23MB/s
1k:5120 Write Speed:1.17MB/s
1k:5120 Write Speed:1.14MB/s
1k:5120 Average Write Speed:1.18MB/s
1k:5120 Read Speed:2.79MB/s
1k:5120 Read Speed:2.80MB/s
1k:5120 Read Speed:2.80MB/s
1k:5120 Average Read Speed:2.79MB/s
Deleting file.
Benchmark done.

The external 3.0 Hard drive benches even faster, with an average sequential read/write of 95MB/s, and random read/write of about 55MB/s.

Someone please post an eyeballed stopwatch test of transferring a giant folder over 70GB (steam games folder would suffice) to a usb 3.0 drive or non-SSD hard drive for a real world test and show how long it takes. sorry for the long post.

Have you checked for firmware updates?

Temperatures and RAM is missing, please add output of

inxi --admin --sensors --info
  System Temperatures: cpu: 30.0 C mobo: N/A
  Fan Speeds (RPM): N/A
  Processes: 228 Uptime: 2d 17h 11m wakeups: 1 Memory: available: 7.68 GiB
  used: 5.13 GiB (66.8%) Init: systemd v: 252 default: graphical
  tool: systemctl Compilers: gcc: 12.2.1 clang: 15.0.7 Packages: pm: pacman
  pkgs: 1215 libs: 330 tools: pamac,yay pm: flatpak pkgs: 0 Shell: Zsh (sudo)
  v: 5.9 default: Bash v: 5.1.16 running-in: konsole inxi: 3.3.27

No firmware updates applicable to anything important except “intel management engine might leak your data to an attacker.” :slight_smile:

No, I didn’t miss it. But unfortunately, something that a lot of people either don’t know or, sometimes conveniently, forgets, is that Windows is not Linux. Windows does things very differently. There’s a lot more donee to make it “idiot proof” and in so doing causes mindsets to become “idiotized” (is that even a word? Apparently so.) Linux is a much more hands-on, no-nonsense and real operating system.

But I’m not here to go on about that. Nor am I here to insult anyone or anything. (Even if it doesn’t look that way.)

Yes and yes, but the limits are software, not hardware. And I’m guessing if that was thee case, there’d be mention of it somewhere in the logs.

But, as I said, I’m not here to insult anyone, so I’ll be off now. TYVM and :v:

:bangbang: Tip: :bangbang:

When posting terminal output, copy the output and paste it here, wrapped in three (3) backticks, before AND after the pasted text. Like this:

pasted text

Or three (3) tilde signs, like this:

pasted text

This will just cause it to be rendered like this:

sollicitudin dolor
eget nisl elit id
arcu erat varius
cursus sem quis eros.

Instead of like this:

Sed sollicitudin dolor eget nisl elit id condimentum arcu erat varius cursus sem quis eros.

Alternatively, paste the text you wish to format as terminal output, select all pasted text, and click the </> button on the taskbar. This will indent the whole pasted section with one TAB, causing it to render the same way as described above.

Thereby increasing legibility thus making it easier for those trying to provide assistance.

For more information, please see:

:bangbang::bangbang: Additionally

If your language isn’t English, please prepend any and all terminal commands with LC_ALL=C. For example:

LC_ALL=C bluetoothctl

This will just cause the terminal output to be in English, making it easier to understand and debug.

No matter how you turn the numbers - the transfer rate is device dependent.

Real world usage never reaches neither specification nor industry tests.

  1. Specification is defining what is possible.
  2. Industry test is using hardware designed to come as close to specification as possible.

The results you can get in real world depends - roughly speaking - on how much money you paid for the device - the good old quantity vs. quality concern.

Most vendors will construct their devices so the apply where the largest demand is - and that is usually Windows.

There is a package in the repo f3 which can check your stick - storage and performance.

$ sudo pacman -Syu f3
$ su -l root
# mount /dev/sdxY /mnt
# f3write /mnt

This one is old - the principle is still valid though

All I’m saying is that I consistently get 50% better speeds (60 vs 40mb/s) under windows vs linux, which would mean the underpinnings either have a bug or are not properly configured/written. 50% is only the benchmark difference. I wouldn’t call that idiot proofing so much as properly engineered. I understand linux doesn’t have that luxury when it comes to reverse engineering or writing from scratch. It’s amazing it works as well as it does considering the lack of support from the industry.

But I’m getting 2 hours to transfer 262GB under windows vs

rsync has been running since about 2 AM here, so 9 hours. Rsync confirms this. It has only transferred 43 GB.

out of the 262 total. This means at the worst it would have to have run for 54 hours on linux, which is unacceptable. That is not a 50% difference. It is a 2600% difference…2days. Worse, it apparently doesn’t affect everyone, making it hard to track down.

Me either and I appreciate your input and all the tips on how to use the forums :sunglasses:

I ran f3 on the first day, sorry I forgot to mention it. I thought it may have been a knockoff flash drive. It checked out okay. I did rerun f3write for you though.

Average writing speed: 39.81 MB/s

Thanks but no thanks on the article about 3.0’s theoretical limits, or theoretical limits in general. I know what those are, which is why I stick to real world examples and benchmarks. This flash stick under performs on benchmarks by half, the hard drive performs worse, and both slow to a crawl to a point of 2 hours vs 2 days. I haven’t posted the results from every machine here, but it affects all of them so far. I have used 2 different external hard drives (the third was SMR, yikes!) and 6 flash drives.

Interestingly I have a sandisk cruzer blade 16GB 2.0 from a few years back that performs pretty normally, on par with Olli’s tests. But then again, it’s only 16GB, so maybe the bug affects it but it just doesn’t run long enough to show it because of the smaller capacity. It bounces between 5-10MB/s which is pretty good.

Mine works fine as is, so it’s not a bug. And I haven’t changed anything regarding the USB or it’s transfers. But then, I might be missing something, since I don’t use any external storage a lot. I used to, but that’s in the past, before I switched to Linux full-time. Now I use the interwebz.

I think you’re misunderstanding things. AFAIK Linux engineers don’t, as you call it or it might actually be, reverse engineer everything. The Kernel is full of drivers, both free and proprietary, as well as some binary blobs, developed by manufacturers. That’s how I have it, anyway. I know the Nouveau Nvidia drivers is free, open source and reverse engineerd, but they don’t work nearly as well as the proprietary ones, and that is a very good example of reversed engineering.

This is, partially at least, true. Linux doesn’t have as big a community as Windows. However, it’s growing. And I don’t know if you know this, but IIRC one of the biggest contributors to the kernel is a developer paid by no one other than Micro$oft itself.*

My reply was not aimed directly at you or your problem. And I’m not accusing anyone of being an idiot, I just mentioned Micro$oft tries to make Windows idiot proof. However, if the shoe fits, by all means put it on. :stuck_out_tongue:

* I’m paraphrasing here, and this might not be :100: at this time. However it is according to my knowledge and understanding so any incorrect information is completely unintentional.