[root tip] [How To] Disable write cache for USB storage devices

Linux filesystem cache

Linux agressively caches files in order to improve overall performance.

When copying large amount of files to an USB storage this often results in some wait time until the device can be safely removed.

How long you have to wait depends on your system and the quality of the USB storage device.

Numerous issues regarding this has resulted in various suggestions involving sysctl tweaks and trim.

Examples:

udev rule to disable write-cache

In another place one user threw in an udev rule which would disable write-cache for devices when they were added and while it was just an idea - it triggered my curiosity.

I dug into the intricacies of udev and found a method to only target USB storage devices.

The rule has gotten a major overhaul and now consist of the rule, a config file and a script

The rule /etc/udev/rules.d/99-usb-sync.rules

# rule to disable write cache for usb storage
# requires hdparm to be installed
ACTION=="add|change", KERNEL=="sd[a-z]", ENV{ID_USB_TYPE}=="disk", RUN+="/usr/bin/hdparm -W 0 /dev/%K"

#
# the following rules is introduced with kernel 6.2
# https://docs.kernel.org/admin-guide/abi-testing.html#abi-sys-class-bdi-bdi-strict-limit
# https://docs.kernel.org/admin-guide/abi-testing.html#abi-sys-class-bdi-bdi-max-ratio
# https://docs.kernel.org/admin-guide/abi-testing.html#abi-sys-class-bdi-bdi-max-bytes
ACTION=="add|change", KERNEL=="sd[a-z]", ENV{ID_USB_TYPE}=="disk", RUN+="/usr/bin/udev-usb-sync %k"

The config /etc/usb-dev-sync/udev-usb-sync.conf

# default values 
#use_tweaks=1
#max_bytes=16777216
#max_ratio=50
#strict_limit=1

The script /usr/bin/udev-usb-sync

#!/usr/bin/bash
#
# script to tweak USB storage device filesystem sync
#
# sources /etc/usb-dev-sync/usb-dev-sync.conf
#

use_tweaks=1
max_bytes=16777216
max_ratio=50
strict_limit=1

# read user config
source /etc/udev-usb-sync/udev-usb-sync.conf

if [[ "$use_tweaks" = 0 ]]; then
        exit 0
fi

if [[ -z "$1" ]]; then
        exit 1
fi

echo "$max_bytes" > "/sys/block/$1/bdi/max_bytes"
echo "$max_ratio" > "/sys/block/$1/bdi/max_ratio"
echo "$strict_limit" > "/sys/block/$1/bdi/strict_limit"

The rule activates when udev detects

  • add or change
  • kernel event for disk devices sd[a-z]
  • only if the device environment ID_USB_TYPE==‘disk’
  • run
    • hdparm -W 0 /dev/%k (disable write cache if supported)
    • udev-usb-sync %k
      • applies defaults
      • read config and apply user values
      • if use_tweaks=0 the script exits
      • if use_tweaks=1 the applies the values (default or config)

Create a file in /etc/udev/rules.d/99-usb-sync.rules and paste the rule into it.
Create a file in /etc/udev-usb-sync/udev-usb-sync.conf and paste the default values.
Create a file in /usr/bin/udev-usb-sync and paste the script content.

Install hdparm package.

sudo pacman -Syu hdparm

Reload udev

sudo udevadm control --reload

Then plug an usb device - open in your file manager - copy a huge amout of files to the device - when the copy is done - click eject in the file manager - note how quick the device is ejected.

For those preferring the package manager, I have created a PKGBUILD which will pull the hdparm dependency upon installation.

pamac build udev-usb-sync

Another fine utility script provided by @cscs fine-tunes a number of system parameters with the option to input your own values when the script is run

11 Likes

Thank you for this. I had been having problems with re-purposed 2.5" SATA, non-SSD drives which I have put in USB 3.0 enclosures. I tried using the USB 3 ports on laptop versus my USB 3 hub and that made no difference. Swap settings didn’t help either. Interestingly, I did not have this issue with the commercial type drives (WD My Passport, Toshiba, etc.). On the drives in the enclosure(s) it would choke with the message “Queued” which when I changed a Thunar setting "Transfer files in parallel, it seemed better, but would take a very long time to eject/safely remove the drive. I also tried different formatting(ext4, xfs, etc.) which made no difference.

I just implemented your solution(manually), rebooted and the process was smooth, no issues. I copied over a 15GB .vdi, unmounted/removed and remounted. All is good. This is a preliminary impression, but I feel confidant with this going forward. Thanks again.

JR

Is sync not enough ?

The Upstream URL in the PKGBUILD leads to a 404 error page (probably not public).

It is.

There has been a lot of topics over time on the subject - just search the forum.

This is meant as a simple - just working - substitution for all the tweaking and fiddling with dirty_bytes and whatnot you will find when searching the forum for the subject.

Example of a solution which likely also works - please read the entire thread - to better understand what happens

SUBSYSTEMS=="usb", SUBSYSTEM=="block", ENV{ID_FS_USAGE}=="filesystem", ENV{UDISKS_MOUNT_OPTIONS_DEFAULTS}+="sync", ENV{UDISKS_MOUNT_OPTIONS_ALLOW}+="sync"

Decrease dirty bytes for more reliable USB transfer - #4 by megavolt

I forgot to create the upstream repo :man_facepalming:

Very nice tutorial :+1:

For those that want to disable caching for only a specific USB-drive, you could use what they explain here: How can I change the cache mode of a USB drive @askubuntu.

  • See also: man udisks

I think I’ve come up with a real solution to this problem that can be evaluated to come by default in Manjaro, posted in another topic about this same problem:

1 Like

There is likely multiple ways of achieving this - what makes you think your solution is more real than other methods?

Why does the solution I presented do what every place I’ve seen addressing this topic says it’s not possible to do, specifying a different cache size in ram for each device.

I used a udev rule that defines 16 MB of cache in ram for any storage device plugged into a USB port, as long as it is not the device where the / system partition is located

It doesn’t change dirty pages of the system as a whole, it doesn’t force sync, it just changes a configuration that is equivalent to dirty pages, but that is specific for each device.

And it’s simple to customize with other cache values or by making modifications to the udev check.

1 Like

My original topic doesn’t deal with dirty pages or anything else - there is a reference to a topic but that is merely reminder that we have seen these topics from time to time and they always end with various tuning suggestions which always is a matter of system and use case.

What I suggest is a simple disable write-cache for the added device if it is an usb disk.

Your suggestion is not bad but I still don’t see why your suggestion is

Your repo was created March 11. - this topic is from March 4. - are you trying to promote something?

I am not critizing your suggestion, I merely curious as to why you think your solution is superior?

Does it mean that the idea presented in the OT or the idea presented by @megavolt is less real?

It is just another way of reaching the goal of having the data written and when no more data - we are done and the device can be ejected.

I don’t know how you interpret the term real solution, English is not my native language, but the difference I looked for when making the solution I presented was to find a method that could be installed in the distribution by default and solve the problem in a way that is useful for almost all users.

The solutions presented before generate effects that, in my point of view, prevent them from being applied in the system.

Can you be more precise about what you’re talking about?

I don’t think a one solution fits all exist.

Your idea and implementation of the idea is great.

The earlier discussions has always ended with some system tweaking which always addressed the system as a whole and thus not targeting removable USB media.

In that regard you are correct - all the previous topics did not address directly USB devices - as it should have.

This is also why I created this topic - a simple method to address the single specific usb attached storage device.

My idea - which was not mine at all - I just expanded on it to only target USB devices and using hdparm to disable write-cache.

Your idea is change the dirty_pages to a size minimal enough to flush quickly - also for the given USB device.

Both works - which one you decide to use - I don’t care - I had my greatest time figuring it out - passing on the knowledge is the least of it.


Perhaps I can enhance your understanding of a solution and a real solution.

A solution could be a work-around - a monkey patch - a bandaid - name it what you like - what matters is that the solution work - but it doesn’t really handle the root cause.

Compared to the above example the real solution is the one which handles the root cause.

We cannot change how the kernel works - but we can apply some rules as how different situations should be handled.

When you look at it that way - there may be multiple options available to achieve the end goal - to be able to remove the USB device immediately when the copy process ends - not waiting - perhaps several mintes before the stick can be removed.

To that end you idea and the idea I grabbed and worked with is equically good.

Hi! Thanks for your hard work and research, I really like the simplicity of this udev rule, but for some reason it seems to not be properly working for me. After creating the rule as described, I enabled logging to check to see if the rule is actually ran:

Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: /etc/udev/rules.d/99-usb-sync.rules:3 RUN '[[ $(uname -r | awk -F'.' '{print $1 $2}') < 62 ]] && [[ $(which hdparm) =~ (hdparm) ]] && /usr/bin/hdparm -W 0 /dev/%k'
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: /etc/udev/rules.d/99-usb-sync.rules:9 RUN '/usr/bin/echo 1 > /sys/block/%k/bdi/strict_limit'
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: /etc/udev/rules.d/99-usb-sync.rules:9 RUN '/usr/bin/echo 50 > /sys/block/%k/bdi/max_ratio'
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: /etc/udev/rules.d/99-usb-sync.rules:9 RUN '/usr/bin/echo 16777216 > /sys/block/%k/bdi/max_bytes'
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Setting permissions /dev/sdd, uid=0, gid=995, mode=0660
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Successfully created symlink '/dev/disk/by-diskseq/27' to '/dev/sdd'
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Successfully created symlink '/dev/disk/by-path/pci-0000:15:00.0-usb-0:2:1.0-scsi-0:0:0:0' to '/dev/sdd'
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Successfully created symlink '/dev/disk/by-id/usb-SanDisk_Cruzer_20043515131DE8F0A85F-0:0' to '/dev/sdd'
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Successfully created symlink '/dev/block/8:48' to '/dev/sdd'
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: sd-device: Created db file '/run/udev/data/b8:48' for '/devices/pci0000:00/0000:00:02.1/0000:02:00.0/0000:03:0c.0/0000:15:00.0/usb3/3-2/3-2:1.0/host12/target12:0:0/12:0:0:0/block/s>
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Running command "[[ $(uname -r | awk -F'.' '{print $1 $2}') < 62 ]] && [[ $(which hdparm) =~ (hdparm) ]] && /usr/bin/hdparm -W 0 /dev/sdd"
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Starting '[[ $(uname -r | awk -F'.' '{print $1 $2}') < 62 ]] && [[ $(which hdparm) =~ (hdparm) ]] && /usr/bin/hdparm -W 0 /dev/sdd'
Aug 25 11:37:41 odin (udev-worker)[10420]: Successfully forked off '(spawn)' as PID 10433.
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Process '[[ $(uname -r | awk -F'.' '{print $1 $2}') < 62 ]] && [[ $(which hdparm) =~ (hdparm) ]] && /usr/bin/hdparm -W 0 /dev/sdd' failed with exit code 1.
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Command "[[ $(uname -r | awk -F'.' '{print $1 $2}') < 62 ]] && [[ $(which hdparm) =~ (hdparm) ]] && /usr/bin/hdparm -W 0 /dev/sdd" returned 1 (error), ignoring.
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Running command "/usr/bin/echo 1 > /sys/block/sdd/bdi/strict_limit"
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Starting '/usr/bin/echo 1 > /sys/block/sdd/bdi/strict_limit'
Aug 25 11:37:41 odin (udev-worker)[10420]: Successfully forked off '(spawn)' as PID 10434.
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: '/usr/bin/echo 1 > /sys/block/sdd/bdi/strict_limit'(out) '1 > /sys/block/sdd/bdi/strict_limit'
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Process '/usr/bin/echo 1 > /sys/block/sdd/bdi/strict_limit' succeeded.
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Running command "/usr/bin/echo 50 > /sys/block/sdd/bdi/max_ratio"
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Starting '/usr/bin/echo 50 > /sys/block/sdd/bdi/max_ratio'
Aug 25 11:37:41 odin (udev-worker)[10420]: Successfully forked off '(spawn)' as PID 10435.
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: '/usr/bin/echo 50 > /sys/block/sdd/bdi/max_ratio'(out) '50 > /sys/block/sdd/bdi/max_ratio'
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Process '/usr/bin/echo 50 > /sys/block/sdd/bdi/max_ratio' succeeded.
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Running command "/usr/bin/echo 16777216 > /sys/block/sdd/bdi/max_bytes"
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Starting '/usr/bin/echo 16777216 > /sys/block/sdd/bdi/max_bytes'
Aug 25 11:37:41 odin (udev-worker)[10420]: Successfully forked off '(spawn)' as PID 10436.
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: '/usr/bin/echo 16777216 > /sys/block/sdd/bdi/max_bytes'(out) '16777216 > /sys/block/sdd/bdi/max_bytes'
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Process '/usr/bin/echo 16777216 > /sys/block/sdd/bdi/max_bytes' succeeded.
Aug 25 11:37:41 odin (udev-worker)[10420]: sdd: Adding watch on '/dev/sdd'


/etc/udev/rules.d ❯ cat /sys/block/sdd/bdi/strict_limit                                                                                                                         
0
/etc/udev/rules.d ❯ cat /sys/block/sdd/bdi/max_ratio                                                                                                                            
100
/etc/udev/rules.d ❯ cat /sys/block/sdd/bdi/max_bytes                                                                                                                            
4462813184

/etc/udev/rules.d ❯ inxi                                                                                                                                                        
CPU: 8-core AMD Ryzen 7 7700X (-MT MCP-) speed/min/max: 3210/3000/5572 MHz
Kernel: 6.4.9-1-MANJARO x86_64 Up: 1h 6m Mem: 10.04/31.06 GiB (32.3%)
Storage: 13.65 TiB (27.7% used) Procs: 416 Shell: Zsh inxi: 3.3.29

It seems the hdparm command fails for some reason, and I do have hdparm installed. But the second set of commands to write to /sys/block seems to succeed, but when I read those files after the values are not written to them. I am on kernel 6.4 so I’d imagine that part of the rule should work. Any idea what’s going wrong here?

I have recently modified the rule for hdparm as I wanted to make sure the rule was only executed when hdparm is actually installed and the kernel is < than 62

This is why the hdparm doesn’t execute - the modification prevent it if kernel is > 6.1

If you want to ensure the rule executes change from

ACTION=="add|change", KERNEL=="sd[a-z]", ENV{ID_USB_TYPE}=="disk", RUN+="[[ $(uname -r | awk -F'.' '{print $1 $2}') < 62 ]] && [[ $(which hdparm) =~ (hdparm) ]] && /usr/bin/hdparm -W 0 /dev/%k"

To

ACTION=="add|change", KERNEL=="sd[a-z]", ENV{ID_USB_TYPE}=="disk", RUN+=" /usr/bin/hdparm -W 0 /dev/%k"
1 Like

Thanks for the reply!

Well either way, shouldn’t the second portion of the rule still be able to function? Any idea of what’s keeping it from successfully writing to /sys/block ?

Changing the rule to what you described and reloading the rules still leaves the USB devices with very long removal times.

Disabling write cache and decreasing the buffer size does not increase the transfer speed to the device - it only ensures that when the copy is done there is no buffer left to flush.

The rest depends on the stick itself.

Ah, but by long removal times I mean it’s reporting the copy as “done” (probably due to misrepresenting the transfer speed) but will not let me safely remove (or eject) the device for sometimes upwards of 10 minutes, depending on the speed of the device or the size of the file(s). I have no intent on trying to speed up the device, I would just like the file transfer to be reported correctly and be able to eject the device after the copy is reported as complete.

Please see some tests I made in a related topic Slow USB transfers - #12 by linux-aarhus