Hello folks,
I know it was controversial in the past what would be the best solution. In file manager you would see a fast transfer speed, then it seems to stuck at 75%. It depends on how much RAM you have and how big your write cache is.
There are 3 approaches:
- Force to use
syncinstead ofasyncas a mount option when using udisks2 aka your file manager, which used it in the background:
SUBSYSTEMS=="usb", SUBSYSTEM=="block", ENV{ID_FS_USAGE}=="filesystem", ENV{UDISKS_MOUNT_OPTIONS_DEFAULTS}+="sync", ENV{UDISKS_MOUNT_OPTIONS_ALLOW}+="sync"
$ dd status=progress if=/dev/urandom of=/run/media/user/USBSTICK/test_sync.img bs=1M count=1000
1046478848 bytes (1,0 GB, 998 MiB) copied, 304 s, 3,4 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1,0 GB, 1000 MiB) copied, 304,593 s, 3,4 MB/s
Need to be noted:
sync
All I/O to the filesystem should be done synchronously. In the case of media with a limited number
of write cycles (e.g. some flash drives), sync may cause life-cycle shortening.
For understanding what it actually does for "Windows users":
While sync is default for usb devices on Windows, it is not on Linux. async is default for any device if not set otherwise.
- Lower/restrict globally can optimize it a bit, but affects all block devices, which is not optimal. It also writes fast and slow down after a while, but slight smoother.
Settings from maxperfwiz, when using 32GB RAM:
vm.dirty_bytes=419430400
vm.dirty_background_bytes=209715200
vm.dirty_expire_centisecs=3000
vm.dirty_writeback_centisecs=1500
vm.min_free_kbytes=311132
$ dd status=progress if=/dev/urandom of=/run/media/user/USBSTICK/test_dirty.img bs=1M count=1000 && time sync
1045430272 bytes (1,0 GB, 997 MiB) copied, 130 s, 8,0 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1,0 GB, 1000 MiB) copied, 130,642 s, 8,0 MB/s
Sure, it is about 3x faster compared to ~300sec, but the sync time need to be added:
sync 0,00s user 0,01s system 0% cpu 1:07,53 total
So it is about 131s + 68s ~ 199s. Still faster than the first method with the sync mount option, but you cannot eject the flash drive immediately and have to wait 1 minute, that is not ideal.
- Using
Backing Device Information(BDI) subsystem for redefining the values per device:
Actually used for fuse or network devices, which are commonly slower, it perfectly fits now slower USB devices.
I followed this: [root tip] [How To] Disable write cache for USB storage devices
$ dd status=progress if=/dev/urandom of=/run/media/user/USBSTICK/test_bdi.img bs=1M count=1000 && time sync
1045430272 bytes (1,0 GB, 997 MiB) copied, 244 s, 4,3 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1,0 GB, 1000 MiB) copied, 244,465 s, 4,3 MB/s
sync 0,00s user 0,02s system 29% cpu 0,076 total
A slight speed up by ~50s compared to the sync option, and you can immediately unplug the flash drive. While still, the async ~50sec faster.
- Without any optimizations:
$ dd status=progress if=/dev/urandom of=/run/media/user/USBSTICK/test_none.img bs=1M count=1000 && time sync
1044381696 bytes (1,0 GB, 996 MiB) copied, 107 s, 9,8 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1,0 GB, 1000 MiB) copied, 107,697 s, 9,7 MB/s
sync 0,00s user 0,01s system 0% cpu 1:13,58 total
107s + 74s = 181s
In simple words:
| Method | total time | sync time |
|---|---|---|
| sync mount option | ~300s | ~0s |
| BDI restriction | ~254s | ~0s |
| optimized dirty pages | ~199s | ~68s |
| no optimizations | ~182s | ~74s |
I know, I know, these tests don’t reflect every setup, but in general it is a test of all methods with a slow device (actually write speed ~4MB/s). async is about 1/3 faster in total, but the sync time is too high for a flash drive. In that case, a sync was forced, imagine how long it takes when it isn’t forced.
I would really see that a 0s sync time would be the default for Manjaro for every usb device. Either the sync option or the BDI limitation. The BDI limitation would certainly be a better choice.
I would argue by saying that flash drives are hot plug devices that can be unplugged immediately. Even professionals sometimes forget this and forget that there is still writing going on in the background what leads to data corruptions.
Better security than performance by default when it comes to data transfers.
Probably add an option to manjaro-settings-manager would ideal, where you can set this option like an on/off toggle.
Thanks for reading.
References:
Here, we cannot do anything in that case.