Slow USB transfers

Support
61

that might be and i’ll check later. but as long as i’m testing now there are terrible results on performance of the preconfigured and in my point of view misconfigurations that lead into missing and bad performance.
the same hardware and meanwhile the usb-transfer exceeded by 300% ! it’s not bit-peeping if a preconfigured linux comes with that bad performance. i do agree the TO’s critics. the overall performance from linux out-of-the box is terrible bad.

p.s.: as side effect the average temperature of the system dropped from 85°Celsius to 64°Celsius. (using actual kernel 6.xx series but 5.xx wasn’t better) . but by digging into the system more and more i realize a lot of crap that is preinstalled and total nonsense.

62

So I have done back to back testing, both benchmarking and real world tests, and have found something. :grinning: I’m not sure what the difference is, because I’m not by any means a super hacker, but out of all the distros I’ve tested:
Kubuntu
Ubuntu
ZorinOS
Manjaro
EndeavourOS
Linux Mint
Fedora
FerenOS
(and a few others lost in the mix…OpenSuse? Been a week)

FerenOS does not seem to be affected by the issue. I have no idea what the difference is considering it is a Ubuntu spinoff. Perhaps someone else can compare the internals and configs?
Both rsync and drag&drop via file managers work fine, and at proper speeds. :slight_smile:

The EXACT SAME folder transfer test hits on average 41 MB/s (113GB transferred to USB 3.0 flash, the same secondary test I’ve devised to test and used over and over at this point…) and completes on par with windows in about 46 minutes running this test. A bit slower by a few minutes, but close. No drops to <4MB/s with hours and hours remaining which occurs on all other distros I’ve tested. I’ve used LiveISO’s, and haven’t changed anything other than the udev rules. Both with and without it, it doesn’t fix the <4MB/s drop issue.

Doing the same folder transfer to the same external 3.0 mechanical hard drive gave similar results on par with windows. 260GB folder took about an hour and a half. Again, proper speeds. I found an external SMR drive as well, and it too only took about an hour and 45 minutes. Pretty darn good.

Way better than 54 hours or longer. :eyes:

It may not be a kernel bug, but it definitely is something that persists across distros.
So, to rules things out:

it’s not the filesystem,
it’s not thermals/error correction,
it’s not a hardware limitation,
it’s not usb devices interfering with each other’s bus communication
it’s not the type of storage medium,
it’s not out of date firmware,
it’s not old hardware,
it’s not the dirty bytes,
it’s not settings I’ve changed (haven’t changed any, it’s a liveISO)
it’s not “windows has proprietary magic,”
benchmarking a particular speed doesn’t mean the slowdown won’t happen,
and best of all it can be tested and verified using real world tests such as large folder transfers.

Whatever FerenOS is doing different is beyond me, but that’s where a clue might be hiding. Thanks again everyone and sorry for the frustration. :upside_down_face:

@Xephon was correct, it does affect certain users and has been lurking under the radar for years. After extensive testing, I can conclusively say whatever is causing it still exists as I remembered it and should be reported to whoever needs to know.

1 Like
63

hello @zippydippy

can you check

lsblk -t

and post the output of feren-os. i’m interested what type of scheduler they use. my output with manjaro is

lsblk -t

NAME   ALIGNMENT MIN-IO OPT-IO PHY-SEC LOG-SEC ROTA SCHED       RQ-SIZE  RA WSAME
sda            0   4096      0    4096     512    0 mq-deadline      64 128    0B
├─sda1         0   4096      0    4096     512    0 mq-deadline      64 128    0B
└─sda2         0   4096      0    4096     512    0 mq-deadline      64 128    0B
sdb            0    512      0     512     512    1 bfq               2 128    0B
sdc            0    512      0     512     512    1 bfq               2 128    0B
├─sdc1         0    512      0     512     512    1 bfq               2 128    0B
└─sdc5         0    512      0     512     512    1 bfq               2 128    0B
sr0            0    512      0     512     512    1 mq-deadline      64 128    0B

where the external usb-devices are controlled via the bfq-scheduler.

64

So if one linux is affected and other is not it is probably down to udev or kernel config and parameters.

65

My exact thoughts. Perhaps get its .configure file and compare it to Manjaros? It’ll be in:

/usr/src/linux-<kernelVersion>/.config

And I believe comparing them is relatively simple with the diff tool.

66

FerenOS is based on Ubuntu 20.04. Did you try Ubuntu 22.04 (current LTS) or 20.04?

67

checking feren os as @zippydippy recommended i can confirm the results of @zippydippy. the speed at large file transfers are doubled in comparison to manjaro. i was already suspicious and now i’m 100% sure that the schedule-handling of manjaro and a lot of other distros are handling external devices with a bad performance. the main difference of feren in comparison is that they use the mq-deadline scheduler for all devices (including external storages) while manjaro uses the bfq-scheduler that doesn’t perform well (if not to say: the bfq-handling is scrap).

lsblk -t

NAME   ALIGNMENT MIN-IO OPT-IO PHY-SEC LOG-SEC ROTA SCHED       RQ-SIZE  RA WSAME
loop0          0    512      0     512     512    1 none            128 128    0B
sda            0   4096      0    4096     512    0 mq-deadline      64 128    0B
├─sda1         0   4096      0    4096     512    0 mq-deadline      64 128    0B
└─sda2         0   4096      0    4096     512    0 mq-deadline      64 128    0B
sdb            0    512      0     512     512    1 mq-deadline       2 128    0B
├─sdb1         0    512      0     512     512    1 mq-deadline       2 128    0B
└─sdb2         0    512      0     512     512    1 mq-deadline       2 128    0B
sdc            0    512      0     512     512    1 mq-deadline       2 128    0B
├─sdc1         0    512      0     512     512    1 mq-deadline       2 128    0B
└─sdc5         0    512      0     512     512    1 mq-deadline       2 128    0B
sr0            0    512      0     512     512    1 mq-deadline      64 128    0B
1 Like
68

Nope actually it does use it:

$ lsblk -t
NAME        ALIGNMENT MIN-IO OPT-IO PHY-SEC LOG-SEC ROTA SCHED       RQ-SIZE  RA WSAME
sda                 0   4096      0    4096     512    1 mq-deadline       2 128    0B
sdb                 0    512      0     512     512    1 mq-deadline       2 128    0B
sdc                 0    512      0     512     512    1 mq-deadline       2 128    0B
sdd                 0    512      0     512     512    1 mq-deadline       2 128    0B
sde                 0   4096      0    4096     512    1 mq-deadline       2 128    0B
sdf                 0    512      0     512     512    1 mq-deadline       2 128    0B
├─sdf1              0    512      0     512     512    1 mq-deadline       2 128    0B
├─sdf2              0    512      0     512     512    1 mq-deadline       2 128    0B
├─sdf3              0    512      0     512     512    1 mq-deadline       2 128    0B
└─sdf4              0    512      0     512     512    1 mq-deadline       2 128    0B
nvme0n1             0    512      0     512     512    0 none           1023 128    0B
├─nvme0n1p1         0    512      0     512     512    0 none           1023 128    0B
├─nvme0n1p2         0    512      0     512     512    0 none           1023 128    0B
├─nvme0n1p3         0    512      0     512     512    0 none           1023 128    0B
├─nvme0n1p4         0    512      0     512     512    0 none           1023 128    0B
├─nvme0n1p5         0    512      0     512     512    0 none           1023 128    0B
├─nvme0n1p6         0    512      0     512     512    0 none           1023 128    0B
├─nvme0n1p7         0    512      0     512     512    0 none           1023 128    0B
└─nvme0n1p8         0    512      0     512     512    0 none           1023 128    0B

Probably you have a udev rules which switch HDDs to bfq?

sda - sdd → BTRFS volume on USB3
sde → BTRFS volume on USB3
sdf → vfat,exfat,ext2 on Flash Drive USB2

69

Same here:

$ lsblk -t
NAME        ALIGNMENT MIN-IO OPT-IO PHY-SEC LOG-SEC ROTA SCHED       RQ-SIZE  RA WSAME
sda                 0   4096      0    4096     512    1 mq-deadline      64 128    0B
└─sda1              0   4096      0    4096     512    1 mq-deadline      64 128    0B
sdb                 0   4096      0    4096     512    1 mq-deadline      64 128    0B
└─sdb1              0   4096      0    4096     512    1 mq-deadline      64 128    0B
nvme0n1             0    512      0     512     512    0 none           1023 128    0B
├─nvme0n1p1         0    512      0     512     512    0 none           1023 128    0B
├─nvme0n1p2         0    512      0     512     512    0 none           1023 128    0B
└─nvme0n1p3         0    512      0     512     512    0 none           1023 128    0B

EXT4 all 'round…

I’m guessing that’s why I don’t have the same performance issue…

:man_shrugging:

70

touche, my mistake, bfq was choosen and installed with this “optimization” script at the beginnig of the thread.
conclusion: this script doesn’t solve the problem

1 Like
71

Because users do their own optimizations. Look in /etc/udev/rules.d/ for your modifications. I pretty sure you set it, since mq-deadline is the default.

I guess chatgpt made it real clear what the difference is: https://chat.openai.com/share/d8cadec9-9dc5-4e12-a378-439103340ee0

Summery:

  • bfq → smooth and responsive experience
  • deadline → optimized for real time and servers, where timed I/O’s are preferred
1 Like
72

Hmm…

Could someone explain the 0 vs 1 in the ROTA colum in above outputs from lsblk -t ?

Could that mean the USB is seen as a rotational disk?
Here’s mine:

lsblk -t
NAME        ALIGNMENT MIN-IO OPT-IO PHY-SEC LOG-SEC ROTA SCHED       RQ-SIZE  RA WSAME
sda                 0    512      0     512     512    0 mq-deadline      64 128    0B
└─sda1              0    512      0     512     512    0 mq-deadline      64 128    0B
sdb                 0    512      0     512     512    1 bfq               2 128    0B
└─sdb1              0    512      0     512     512    1 bfq               2 128    0B
nvme0n1             0    512      0     512     512    0 none            255 128    0B
├─nvme0n1p1         0    512      0     512     512    0 none            255 128    0B
└─nvme0n1p2         0    512      0     512     512    0 none            255 128    0B

sdb is the USB.

73

Yes it is.

Ah I see the flash drive here on my side is detected as rotational disk. Funny :smiley: But I don’t see why… anyway that is offtopic.

74

It’s not rotational, so I wonder…

Edit, found this:
https://bugzilla.kernel.org/show_bug.cgi?id=90761

75

My system also uses mq-deadline for all devices …

 $ lsblk -t
NAME        ALIGNMENT MIN-IO   OPT-IO PHY-SEC LOG-SEC ROTA SCHED       RQ-SIZE    RA WSAME
sda                 0   4096        0    4096     512    0 mq-deadline      64   128    0B
└─sda1              0   4096        0    4096     512    0 mq-deadline      64   128    0B
sdb                 0    512        0     512     512    0 mq-deadline      64   128    0B
└─sdb1              0    512        0     512     512    0 mq-deadline      64   128    0B
sdc                 0    512        0     512     512    0 mq-deadline      64   128    0B
└─sdc1              0    512        0     512     512    0 mq-deadline      64   128    0B
sdd                 0    512        0     512     512    1 mq-deadline       2   128    0B
sde                 0    512        0     512     512    1 mq-deadline       2   128    0B
sdf                 0    512        0     512     512    1 mq-deadline       2   128    0B
sdg                 0   4096        0    4096     512    0 mq-deadline      60   128    0B
├─sdg1              0   4096        0    4096     512    0 mq-deadline      60   128    0B
├─sdg2              0   4096        0    4096     512    0 mq-deadline      60   128    0B
└─sdg3              0   4096        0    4096     512    0 mq-deadline      60   128    0B
sdh                 0   4096        0    4096     512    1 mq-deadline      60   128    0B
└─sdh1              0   4096        0    4096     512    1 mq-deadline      60   128    0B
sdi                 0    512 33553920     512     512    0 mq-deadline      60 65532    0B
└─sdi1              0    512 33553920     512     512    0 mq-deadline      60 65532    0B
sdj                 0    512        0     512     512    1 mq-deadline       2   128    0B
└─sdj1              0    512        0     512     512    1 mq-deadline       2   128    0B
nvme1n1             0    512        0     512     512    0 none           1023   128    0B
└─nvme1n1p1         0    512        0     512     512    0 none           1023   128    0B
nvme0n1             0    512        0     512     512    0 none           1023   128    0B
├─nvme0n1p1         0    512        0     512     512    0 none           1023   128    0B
├─nvme0n1p2         0    512        0     512     512    0 none           1023   128    0B
└─nvme0n1p3         0    512        0     512     512    0 none           1023   128    0B

These are an USB (internal) attached card reader (empty)

sdd                 0    512        0     512     512    1 mq-deadline       2   128    0B
sde                 0    512        0     512     512    1 mq-deadline       2   128    0B
sdf                 0    512        0     512     512    1 mq-deadline
76

I don’t know what VPD BDC is, but according to the link from @Pippin:

Currently, there is no way to distinguish if an USB that don’t have VPD is a flash disk, because VPD BDC is used. USB sticks that have VPD BDC should be set as not rotational as expected. So it’s a known issue, but so far there is no way to fix it.

Might be relevant?

:man_shrugging:

77

Has anyone deduced the difference? I don’t believe its the scheduler or the VPD BDC info, as mine continues to work under FerenOS despite being set to rotational by default.

78

I think we can agree that an old rotational SATA disk 5400rpm is significantly slower than a modern 128G USB flash drive - am I right?

On 2023-08-31T22:00:00Z I did a test on an old ThinkPad T550 using two different rotational disks connected using USB3 enclosure connected to USB3 port.

Some real world copying of folders with thousands of files from tiny sources files to huge zip archives and isos. Sizes range from a few kb to 10GB

The size as reported by du

11:59:33 ○ [fh@panther] ~
 $ echo $(du -h ./test | tail -n 1 | awk '{print $1}' | tr -d '\n')
278G

09:29:37 ○ [fh@panther] ~
 $ pacman-mirrors -G
unstable

09:29:52 ○ [fh@panther] ~
 $ inxi -SCm
System:
  Host: panther Kernel: 6.4.12-1-MANJARO arch: x86_64 bits: 64
    Desktop: KDE Plasma v: 5.27.7 Distro: Manjaro Linux
Memory:
  System RAM: total: 16 GiB available: 15.52 GiB used: 1.55 GiB (10.0%)
  RAM Report: permissions: Unable to run dmidecode. Root privileges
    required.
CPU:
  Info: dual core model: Intel Core i7-5600U bits: 64 type: MT MCP cache:
    L2: 512 KiB
  Speed (MHz): avg: 2746 min/max: 500/3200 cores: 1: 3200 2: 2594 3: 2596
    4: 2594

Old Samsung SpinPoint ST750LM022
From Amazon About this item

  • Samsung SpinPoint ST750LM022 750GB 2.5-inch Hard Drive General Features: 750 GB capacity
  • SATA interface 3 Gbps transfer rate 5400 RPM spindle speed 8 MB buffer
  • 12 ms Average Seek Time (typical) 5.6 ms Average Latency 4 second Drive Ready Time SilentSeek
  • NoiseGuard Lead-free 2.5-inch form factor ATA S.M.A.R.T. feature set ATA security mode feature set
11:56:22 ○ [fh@panther] ~
 $ lsblk -f
NAME   FSTYPE FSVER LABEL          UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
sda                                                                                    
├─sda1 vfat   FAT32                8725-B184                             299,1M     0% /boot/efi
├─sda2 f2fs   1.16                 d519cf49-deb0-466c-9cdc-79ff7125cb1d  170,7G    63% /
└─sda3 swap   1     swap           591812b9-9f86-4a22-82ef-850b7624e6fd                [SWAP]
sdb                                                                                    
└─sdb1 ext4   1.0   seagate-usb    b925148a-0729-447a-ad95-4d9e27dcb442  655,6G    59% /mnt/seagate-usb
sdc                                                                                    
└─sdc1 ntfs         samsung750ntfs 5337D78206CA8C03                      698,5G     0% /mnt/Samsung750ntfs

From local ssd (f2fs) to 5400RPM Seagate BUP Slim 2TB (5400rpm) (ext4) connected to USB-3 port transferred using rsync

12:01:34 ○ [fh@panther] ~
 $ time rsync -av ./test/. /mnt/seagate-usb/
sending incremental file list

[...]

sent 297.219.367.236 bytes  received 723.841 bytes  99.321.667,86 bytes/sec
total size is 297.143.890.573  speedup is 1,00

real    49m51,609s
user    8m22,617s
sys     23m49,808s

From local ssd (f2fs) to Samsung ST750ML022 (5400rpm) (ntfs) connected to USB-3 port using USB-3 enclosure transferred using rsync

13:21:00 ○ [fh@panther] ~
 $ time rsync -av ./test/. /mnt/Samsung750ntfs/
sending incremental file list

[...]

sent 297.219.367.236 bytes  received 724.737 bytes  73.469.309,60 bytes/sec
total size is 297.143.890.573  speedup is 1,00

real    67m25,333s
user    3m24,771s
sys     10m53,567s
79

I cannot comment on the 300GB scenario, but i last copied about 10 GB in some iso files on 2 different flash drives (the same system, kernel 6.1 and the udev rules from the aur packet from linux-aarchus). In that scenario there was a huge difference between the 2 drives.
Intenso alu line for 5 euro was a disaster reaching the staggering 2MB/s after about 5 seconds.
Topsel 128G was at least 10-20 times faster. Didn’t really remember.

So for small amounts it definitely depends of the quality of the drive as written at the beginning of the topic.

80

this is a already known fact, but it doesn’t explain that testing one device does have so extreme different results in speed. this is not because of the device-quality but it is because of the different handling in file-transfers.