[root tip] [How To] Disable write cache for USB storage devices

Linux filesystem cache

Linux agressively caches files in order to improve overall performance.

When copying large amount of files to an USB storage this often results in some wait time until the device can be safely removed.

How long you have to wait depends on your system and the quality of the USB storage device.

Numerous issues regarding this has resulted in various suggestions involving sysctl tweaks and trim.

One example is this topic Decrease dirty bytes for more reliable USB transfer

udev rule to disable write-cache

In another place one user threw in an udev rule which would disable write-cache for devices when they were added and while it was just an idea - it triggered my curiosity.

I dug into the intricacies of udev and found a method to only target USB storage devices.

The final rule

# rule to disable write cache for usb storage
# requires hdparm to be installed
ACTION=="add|change", KERNEL=="sd[a-z]", ENV{ID_USB_TYPE}=="disk", RUN+="/usr/bin/hdparm -W 0 /dev/%k"

The rule activates on

  • add or change
  • kernel event for disk devices sd[a-z]
  • only if the device environment ID_USB_TYPE==‘disk’
  • run hdparm -W 0 /dev/%K

Create a file in /etc/udev/rules.d/99-usb-sync.rules and paste above content into the file and save it.

Install hdparm package.

sudo pacman -Syu hdparm

Then plug an usb device - open in your file manager - copy a huge amout of files to the device - when the copy is done - click eject in the file manager - note how quick the device is ejected.

For those preferring the package manager, I have created a PKGBUILD which will pull the hdparm dependency upon installation.

pamac build udev-usb-sync
8 Likes

Thank you for this. I had been having problems with re-purposed 2.5" SATA, non-SSD drives which I have put in USB 3.0 enclosures. I tried using the USB 3 ports on laptop versus my USB 3 hub and that made no difference. Swap settings didn’t help either. Interestingly, I did not have this issue with the commercial type drives (WD My Passport, Toshiba, etc.). On the drives in the enclosure(s) it would choke with the message “Queued” which when I changed a Thunar setting "Transfer files in parallel, it seemed better, but would take a very long time to eject/safely remove the drive. I also tried different formatting(ext4, xfs, etc.) which made no difference.

I just implemented your solution(manually), rebooted and the process was smooth, no issues. I copied over a 15GB .vdi, unmounted/removed and remounted. All is good. This is a preliminary impression, but I feel confidant with this going forward. Thanks again.

JR

Is sync not enough ?

The Upstream URL in the PKGBUILD leads to a 404 error page (probably not public).

It is.

There has been a lot of topics over time on the subject - just search the forum.

This is meant as a simple - just working - substitution for all the tweaking and fiddling with dirty_bytes and whatnot you will find when searching the forum for the subject.

Example of a solution which likely also works - please read the entire thread - to better understand what happens

SUBSYSTEMS=="usb", SUBSYSTEM=="block", ENV{ID_FS_USAGE}=="filesystem", ENV{UDISKS_MOUNT_OPTIONS_DEFAULTS}+="sync", ENV{UDISKS_MOUNT_OPTIONS_ALLOW}+="sync"

Decrease dirty bytes for more reliable USB transfer - #4 by megavolt

I forgot to create the upstream repo :man_facepalming:

Very nice tutorial :+1:

For those that want to disable caching for only a specific USB-drive, you could use what they explain here: How can I change the cache mode of a USB drive @askubuntu.

  • See also: man udisks

I think I’ve come up with a real solution to this problem that can be evaluated to come by default in Manjaro, posted in another topic about this same problem:

There is likely multiple ways of achieving this - what makes you think your solution is more real than other methods?

Why does the solution I presented do what every place I’ve seen addressing this topic says it’s not possible to do, specifying a different cache size in ram for each device.

I used a udev rule that defines 16 MB of cache in ram for any storage device plugged into a USB port, as long as it is not the device where the / system partition is located

It doesn’t change dirty pages of the system as a whole, it doesn’t force sync, it just changes a configuration that is equivalent to dirty pages, but that is specific for each device.

And it’s simple to customize with other cache values or by making modifications to the udev check.

1 Like

My original topic doesn’t deal with dirty pages or anything else - there is a reference to a topic but that is merely reminder that we have seen these topics from time to time and they always end with various tuning suggestions which always is a matter of system and use case.

What I suggest is a simple disable write-cache for the added device if it is an usb disk.

Your suggestion is not bad but I still don’t see why your suggestion is

Your repo was created March 11. - this topic is from March 4. - are you trying to promote something?

I am not critizing your suggestion, I merely curious as to why you think your solution is superior?

Does it mean that the idea presented in the OT or the idea presented by @megavolt is less real?

It is just another way of reaching the goal of having the data written and when no more data - we are done and the device can be ejected.

I don’t know how you interpret the term real solution, English is not my native language, but the difference I looked for when making the solution I presented was to find a method that could be installed in the distribution by default and solve the problem in a way that is useful for almost all users.

The solutions presented before generate effects that, in my point of view, prevent them from being applied in the system.

Can you be more precise about what you’re talking about?

I don’t think a one solution fits all exist.

Your idea and implementation of the idea is great.

The earlier discussions has always ended with some system tweaking which always addressed the system as a whole and thus not targeting removable USB media.

In that regard you are correct - all the previous topics did not address directly USB devices - as it should have.

This is also why I created this topic - a simple method to address the single specific usb attached storage device.

My idea - which was not mine at all - I just expanded on it to only target USB devices and using hdparm to disable write-cache.

Your idea is change the dirty_pages to a size minimal enough to flush quickly - also for the given USB device.

Both works - which one you decide to use - I don’t care - I had my greatest time figuring it out - passing on the knowledge is the least of it.


Perhaps I can enhance your understanding of a solution and a real solution.

A solution could be a work-around - a monkey patch - a bandaid - name it what you like - what matters is that the solution work - but it doesn’t really handle the root cause.

Compared to the above example the real solution is the one which handles the root cause.

We cannot change how the kernel works - but we can apply some rules as how different situations should be handled.

When you look at it that way - there may be multiple options available to achieve the end goal - to be able to remove the USB device immediately when the copy process ends - not waiting - perhaps several mintes before the stick can be removed.

To that end you idea and the idea I grabbed and worked with is equically good.