Graceful way to spin down (unload head) USB attached mobile hard drive BEFORE shutdown(poweroff/reboot/halt)

wlmqljj · 23 March 2023 07:32

Usage Scenario:
System running on a USB3.0 mobile HDD docker or attached with a external hard drive for more storage.

Issue:
My Erp is ON (Powerdown the USB 5V VBUS after shutdown----motherboard’s function)
External usb scsi HDDs will NOT park their heads to parking lot keeping spinning.
Unmount.target is not enough for usb scsi subsystem.
It powerdown my HDD brutally.
At least It not work for my Asmedia 1153e chip HDD case.

My expect:
Add the udev policy as default or more graceful solutions to solve the issue.
the most graceful way to do this is:

find /sys -name manage_start_stop
echo 1 > /sys/.../manage_start_stop

or udev policy

ACTION=="add|change", DRIVERS=="usb-storage|uas", SUBSYSTEM=="scsi_disk", ATTR{manage_start_stop}="1"

My difficulties(LONG story):
It’s really frustrating and disappointing when I realize my “HDD Emergency Power-Off Retract Count” in SMART INfo have increased to nearly thousands.
Just because there is no hint that "sudo poweroff \ systemctl poweroff \ reboot \ init 0\ init 6 " will NOT properly power off my USB HDD, it only unmount partitions, then it just hard power off.
I tried searching the internet then they all answered with udiskctl, hdparm, udev script, sdparm, manual unmout action in dophin(KDE User), from here\ArchWiki\reddit\google\askubuntu\raspberrypi_forum… just to name a few.
And they all too hard and do not work for me.
It just too hard for me to understand how systemd works, and how to write bash script.
I just want my HDD power off before shutdown safely like Windows or MacOS.
Then I finally notice a post on
bugzilla;kernel;org/show_bug.cgi?id=205107
and followed the comment 21 , then it works.
It takes too long to solve this tiny problem…

THANKS in advance if it is a default in manjaro ( I don’t know other distros have this ).
And if there is better solution please let me know, Appreciated!

Mirdarthos · 23 March 2023 07:43

Hi @wlmqljj, and welcome!

According to this page:

The udisks command is most likely what you are looking for.

While sudo unmount /dev/sdXY will work, udisks can do this without root level (sudo) permissions.

If you have a drive /dev/sdXY, mounted, where X is a letter representing your usb disk and Y is the partition number (usually 1), you can use the following commands to safely remove the drive:
udisks --unmount /dev/sdXY
udisks --detach /dev/sdX
For a practical example, if I have the partition /dev/sdb1 mounted, I would run this to unmount and detach it:
udisks --unmount /dev/sdb1
udisks --detach /dev/sdb
If your drive is not mounted, or was never mounted, simply use the second command:
udisks --detach /dev/sdb
I originally found this through this question: https://superuser.com/a/430470/176493.

Using udisks2:

In the newer ubuntu distributions (I’m unsure of when the switch occurred), udisks2 is installed instead of udisks.

Mirroring the commands above, to unmount and detach a disk with udisks2:
udisksctl unmount -b /dev/sdXY
udisksctl power-off -b /dev/sdX
Example if my drive is /dev/sdb1:
udisksctl unmount -b /dev/sdb1
udisksctl power-off -b /dev/sdb
Similarly to above, power-off can be used to detach the drive even if there are no partitions mounted, or no partition was ever mounted:
udisksctl power-off -b /dev/sdb

Hope this helps!

wlmqljj · 23 March 2023 07:55

Really Appreciated @Mirdarthos [Mirdarthos]
, but udisk2 util udiskctl need to be execution manually or add it as systemd script to autorun.
And seems not work when systemd reach unmount.target for my case for unknown reasons.
And the hdparm sdparm also need a script to do this.

I think these solutions are not graceful and maybe not reliable ,
for those users have mutiple usb scsi HDD ,they need a bash(shell) script to recurse all /dev/sd* to execute the script ,and tracing the execution result is not easy.
The solution above

find /sys -name manage_start_stop
echo 1 > /sys/.../manage_start_stop

or udev

ACTION=="add|change", DRIVERS=="usb-storage|uas", SUBSYSTEM=="scsi_disk", ATTR{manage_start_stop}="1"

is more user frendly and easy to understand.

Mirdarthos · 23 March 2023 08:01

AFAIK nothing special is needed on shutdown, since it’s done automagically.

If you want it do-able manually, then you can do it in a short script which you run instead of remove the device manually, but that is way. way. WAY above my paygrade.

wlmqljj · 23 March 2023 08:07

That’s right,
It should do it automagically, but the result is not,
the post on bugzilla for kenel org show the detailed issue
bugzilla;kernel;org/show_bug.cgi?id=205107
(i can’t post URL,use ; to repace .)

I do not know why linux kernel not do this automagically like windows and macos

linux-aarhus · 23 March 2023 08:08

Doesn’t modern disk implement autopark for the same reason?

I have a Seagate 2TB rotational 5400RPM and I have two choices with the file manager

unmount
safely remove

The first unmount - but do not power down the device
The second unmount and power down the disk - completely removing it from the system.

If I don’t unmount the device but instead I shut down the system the device unmount but stays powered until the hardware is powered off.

I think you point to the cause by the mention of your HDD case - perhaps the case is causing this?

The usage of an external case and disk powered over USB is quite common.

So here is an idea of what is going on.

When you use the same disk inside your system connected to the mainboard using sata or similar your disk will receive a command to park heads prior to power down.

The same signal may be send to the USB connected device but the case interface may not pass on the signal.

wlmqljj · 23 March 2023 08:15

Yes ,the HDD will auto park after a period of time,
But I have Erp on(Powerdown the USB 5V VBUS after shutdown----motherboard’s function)
The mother board will turnoff 5v vbus for eco (I don’t want my perpheral lights on).

for USB SCSI DISK shutdown seems it not sending SCSI spin down command unless "
/sys/…/manage_start_stop" is set to 1,
It just unmount,
only SCSI read/write USB packets is transfered when systemd handle unmount.target.And the disk is not prepared to be turnoff.

The asmedia 1153e and other WD passport is the issue indeed ,
but the implement of linux driver to handle USB SCSI DISK shutdown spindown is not default.
It is default for windows, every reboot shutdown hibernate or sleep , it spin down the disks before system call acpi to shut the mother board vbus.
the linux need to be done manully.

linux-aarhus · 23 March 2023 08:24

I see - I have a vague recollection that I have heard about this once perhaps twice the past 10 years.

Perhaps we should create a short guide in the Tutorials section

wlmqljj · 23 March 2023 08:49

That’s right. It’s an old issue.
I’m not familar with linux kernel dev,
Maybe manually turn off the usb disk is right solution and it’s the creed for geek linux user.
But it will be really disappointing when I found my usb HDD broken because of this.

Related Doc:

github.com

torvalds/linux/blob/v5.11/include/scsi/scsi_device.h#L186


      
          	unsigned no_report_opcodes:1;	/* no REPORT SUPPORTED OPERATION CODES */
          	unsigned no_write_same:1;	/* no WRITE SAME command */
          	unsigned use_16_for_rw:1; /* Use read/write(16) over read/write(10) */
          	unsigned skip_ms_page_8:1;	/* do not use MODE SENSE page 0x08 */
          	unsigned skip_ms_page_3f:1;	/* do not use MODE SENSE page 0x3f */
          	unsigned skip_vpd_pages:1;	/* do not read VPD pages */
          	unsigned try_vpd_pages:1;	/* attempt to read VPD pages */
          	unsigned use_192_bytes_for_3f:1; /* ask for 192 bytes from page 0x3f */
          	unsigned no_start_on_add:1;	/* do not issue start on add */
          	unsigned allow_restart:1; /* issue START_UNIT in error handler */
          	unsigned manage_start_stop:1;	/* Let HLD (sd) manage start/stop */
          	unsigned start_stop_pwr_cond:1;	/* Set power cond. in START_STOP_UNIT */
          	unsigned no_uld_attach:1; /* disable connecting to upper level drivers */
          	unsigned select_no_atn:1;
          	unsigned fix_capacity:1;	/* READ_CAPACITY is too high by 1 */
          	unsigned guess_capacity:1;	/* READ_CAPACITY might be too high by 1 */
          	unsigned retry_hwerror:1;	/* Retry HARDWARE_ERROR */
          	unsigned last_sector_bug:1;	/* do not use multisector accesses on
          					   SD_LAST_BUGGY_SECTORS */
          	unsigned no_read_disc_info:1;	/* Avoid READ_DISC_INFO cmds */
          	unsigned no_read_capacity_16:1; /* Avoid READ_CAPACITY_16 cmds */

http://www.linux-ata.org/shutdown.html

https://bugzilla.kernel.org/show_bug.cgi?id=205107

Solutions:

add a udev rule

/etc/udev/rules.d/01-manage_start_stop.rules

ACTION=="add|change", DRIVERS=="usb-storage|uas", SUBSYSTEM=="scsi_disk", ATTR{manage_start_stop}="1"

do it Manually (do it as super user)

find /sys -name manage_start_stop
echo 1 > /sys/class/scsi_disk/h:c:i:l/manage_start_stop
("h:c:i:l" replace to what you saw above)

linux-aarhus · 23 March 2023 08:58

I read it to be the default for devices in sd tree?

wlmqljj · 23 March 2023 09:03

But i do need set it to be 1 to make it work.
I have not figured out why.
I cat my usb scsi device as 0 by default:
cat /sys/devices/pci0000:00/0000:00:08.1/0000:2a:00.3/usb4/4-1/4-1:1.0/host10/target10:0:0/10:0:0:0/scsi_disk/10:0:0:0/manage_start_stop

0

Maybe linux kernel do not handle USB SCSI Device like other native ones

wlmqljj · 15 October 2024 03:05

After some digging into the code.
I probably find the reason why the usb-storage, uas, and some sas driver do not handle this.
Driver need to operate generic devices, maybe the chip itself handling spindown must be the proper way. Sadly, chip manufactures often have their unique quirks.
Such as this Asmedia chip asm1135.
But if Chips do not handle this as a routine, then the driver should take responsibility to tackle with it.
I found the commit Damien Le Moal did last year for Firewire. It clearly show this type of device configuration need to be done within low level driver.

github.com

torvalds/linux/blob/eca631b8fe808748d7585059c4307005ca5c5820/drivers/firewire/sbp2.c#L1509-L1539


      
          static int sbp2_scsi_device_configure(struct scsi_device *sdev,
          		struct queue_limits *lim)
          {
          	struct sbp2_logical_unit *lu = sdev->hostdata;
          
          	sdev->use_10_for_rw = 1;
          
          	if (sbp2_param_exclusive_login) {
          		sdev->manage_system_start_stop = 1;
          		sdev->manage_runtime_start_stop = 1;
          		sdev->manage_shutdown = 1;
          	}
          
          	if (sdev->type == TYPE_ROM)
          		sdev->use_10_for_ms = 1;
          
          	if (sdev->type == TYPE_DISK &&
          	    lu->tgt->workarounds & SBP2_WORKAROUND_MODE_SENSE_8)
          		sdev->skip_ms_page_8 = 1;

This file has been truncated. show original

So as to usb-storage and uas, need to add

sdev->manage_runtime_start_stop = 1;
sdev->manage_shutdown = 1;

as well in their configuration.
For usb-storage:

github.com

torvalds/linux/blob/eca631b8fe808748d7585059c4307005ca5c5820/drivers/usb/storage/scsiglue.c#L91-L329


      
          static int device_configure(struct scsi_device *sdev, struct queue_limits *lim)
          {
          	struct us_data *us = host_to_us(sdev->host);
          	struct device *dev = us->pusb_dev->bus->sysdev;
          
          	/*
          	 * Many devices have trouble transferring more than 32KB at a time,
          	 * while others have trouble with more than 64K. At this time we
          	 * are limiting both to 32K (64 sectores).
          	 */
          	if (us->fflags & (US_FL_MAX_SECTORS_64 | US_FL_MAX_SECTORS_MIN)) {
          		unsigned int max_sectors = 64;
          
          		if (us->fflags & US_FL_MAX_SECTORS_MIN)
          			max_sectors = PAGE_SIZE >> 9;
          		lim->max_hw_sectors = min(lim->max_hw_sectors, max_sectors);
          	} else if (sdev->type == TYPE_TAPE) {
          		/*
          		 * Tapes need much higher max_sector limits, so just
          		 * raise it to the maximum possible (4 GB / 512) and

This file has been truncated. show original

For uas:

github.com

torvalds/linux/blob/eca631b8fe808748d7585059c4307005ca5c5820/drivers/usb/storage/uas.c#L835-L901


      
          static int uas_device_configure(struct scsi_device *sdev,
          		struct queue_limits *lim)
          {
          	struct uas_dev_info *devinfo = sdev->hostdata;
          
          	if (devinfo->flags & US_FL_MAX_SECTORS_64)
          		lim->max_hw_sectors = 64;
          	else if (devinfo->flags & US_FL_MAX_SECTORS_240)
          		lim->max_hw_sectors = 240;
          
          	if (devinfo->flags & US_FL_NO_REPORT_OPCODES)
          		sdev->no_report_opcodes = 1;
          
          	/* A few buggy USB-ATA bridges don't understand FUA */
          	if (devinfo->flags & US_FL_BROKEN_FUA)
          		sdev->broken_fua = 1;
          
          	/* UAS also needs to support FL_ALWAYS_SYNC */
          	if (devinfo->flags & US_FL_ALWAYS_SYNC) {
          		sdev->skip_ms_page_3f = 1;

This file has been truncated. show original

Other driver also need similar change to adapt Damien Le Moal changed last year.

I did made some mod in kernel for my own use. But I have never post a patch to kernel mailing list, I think it is not a easy thing to start.
So if someone is experienced and willing to make a patch for upstream kernel driver, I will be appreciated!

For average user who using kernel version 6.1 and beyond, as Harddisk spin down on shutdown no longer works mentioned.
Apply udev rule

ACTION=="add|change", DRIVERS=="usb-storage|uas", SUBSYSTEM=="scsi_disk", ATTR{manage_runtime_start_stop}="1", ATTR{manage_shutdown}="1"

is enough according to linux/drivers/ata/libata-scsi.c at eca631b8fe808748d7585059c4307005ca5c5820 · torvalds/linux · GitHub

For user using kernel before 6.1, apply the old udev rule.

ACTION=="add|change", DRIVERS=="usb-storage|uas", SUBSYSTEM=="scsi_disk", ATTR{manage_start_stop}="1"