Awesome. I’ll plug up the USB hub in a minute and see if it remains stable for 24 hours with nothing plugged in.
EDIT 2: Anecdotally, USB NVME disk read now seems more efficient/faster. It’s hitting the 300 MB/s max more easily. I rarely saw anything over 285 before.
EDIT 1: It’s 11 PM US Central Time on March 17, 2021. The USB hub is plugged in, and visible to Pi. I’m not going to use it, and just use the Pi as normal. If the Pi doesn’t crash in 24 hours, I’ll assume it’s stable and start experimenting with plugging things in.
I added some additional info to my earlier post, just to document the parts of the firmware release notes that seemed to be related to NVME/USB boot.
A couple of questions:
- Do I need to be concerned about that “start.elf update” remark? I’m assuming the packages I downloaded updated the .elf file and anything else necessary, since the Pi actually, y’know, booted again.
- “Improve compatibility with external USB 3.0 disk enclosures by enumerating the downstream hubs before executing the USB port power off.” Any idea what this means? Specifically, “enumerating the downstream hubs?” I can half-guess, but I’m probably way off the mark.
EDIT: I plugged the NVME SSD enclosure into my powered hub and started up the Pi. It boots. @Darksky , I think this is a definite improvement. I could not get it to reliably boot from the SSD when it was plugged into the hub before. I’m very pleased with this, as I was concerned about the Pi being able to adequately power the NVME over the bus.
On the downside, the hub does appear to introduce some significant overhead. I’m seeing a 15-25 percent read speed drop in hdparm vs plugging the drive directly into the Pi. The slowest I’m seeing is still ~245MB/s, which is still leagues faster than the internal SD card (~45 MB/s) or the onboard 1Gbps Ethernet.
EDIT 2: I ran into some serious issues when I tried to shut down the Pi so I could unplug the onboard Ethernet. It absolutely refused to come back up. No disk activity at all while the SSD was plugged into the hub. I removed the hub from the Pi and plugged the ethernet adapter in directly…and it had put itself back into USB storage mode (for installing drivers when plugged into a Windows machine).
Bus 001 Device 003: ID 0bda:8151 Realtek Semiconductor Corp. RTL8151 Adapteon Business Mobile Networks BV
This command switches its mode to ethernet adapter:
sudo usb_modeswitch -v 0bda -p 8151 -V 0bda -P 8152 -M 555342430860d9a9c0000000800006e0000000000000000000000000000000
However, the mode switch does not survive restart. I’m going to have to look at a udev rule, I think, which I don’t completely understand how to write. I know there’s a way to automatically run that command on startup with Udev, though.
EDIT 3: I bounced around google last night until I found someone who was trying to do a modes witch on their USB device via Udev, and wasn’t also trying to do sixteen other things at once, so I was actually able to figure it out.
Here’s the Udev rule:
❯ cat /etc/udev/rules.d/40-usb-realtek-modeswitch.rules
ACTION=="add|change", SUBSYSTEM=="usb", ATTRS{idVendor}=="0bda", ATTRS{idProduct}=="8151", RUN+="/usr/sbin/usb_modeswitch -v 0bda -p 8151 -V 0bda -P 8152 -M '555342430860d9a9c0000000800006e0000000000000000000000000000000'"
Here’s the mode-switched device:
❯ lsusb
Bus 001 Device 004: ID 0bda:8156 Realtek Semiconductor Corp. USB 10/100/1G/2.5G LAN
After making sure the right usb device name is used in ‘nmtui’, the interface comes up:
❯ ip addr show enp1s0u1u1
4: enp1s0u1u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether $HW_MAC brd ff:ff:ff:ff:ff:ff
inet 10.0.4.51/24 brd 10.0.4.255 scope global dynamic noprefixroute enp1s0u1u1
valid_lft 86108sec preferred_lft 86108sec
Conclusion:
-
The EEPROM update + the patched kernel did the trick. Now I’ve got the 2.5GbE dongle and the NVME SSD on the same USB bus. This still worked after shutting down the Pi, removing the ethernet cable from the onboard 1GbE, and restarting it.
-
LightDM is non-functional again. I have no idea what’s going on here, aside from the fact Xorg refuses to see any screens plugged in using this kernel. I will be pleased when I can finally switch to X11 forwarding.
-
I’m going avoid messing with the network settings for at least 12 hours. If anything is going to become unstable and crash, it shouldn’t take longer than that.
Questions:
- Is there anything specific you’d like me to test, aside from it just not crashing?
- You mentioned in the Kernel thread that you were rolling this change into the latest unstable 5.10.y kernel a while ago. Does that mean that at some point, assuming all this works, I’ll be able to install the latest 5.10.y unstable kernel and insert this altered driver into it?
Update: It crashed again, roughly 2 minutes after I updated this post to say it worked. While the new firmware definitely improved things–the SSD and ethernet adapter were stable on the same USB bus for minutes before it all went boom–it’s still apparently hardware limited.
It didn’t seem to make a difference whether the SSD or the USB ethernet adapter was plugged directly into the Pi, while the other device was plugged into the powered hub.
I need to get back to work, but later on I’ll do two more tests:
-
Try plugging the ethernet adapter in to the USB 3.0 port after the Pi has successfully booted off the SSD. It may be possible that something is going on specific to trying to boot with it, given that the one time it seemed to work was when I plugged it in after the Pi was already booted. If that’s the case, maybe
dmesg will reveal something useful about what’s going on. But it’s not really feasible, IMHO, to use an ethernet adapter that can’t be plugged in when the system boots–especially given how often I end up rebooting the Pi while tweaking various docker containers and other things actually involving using the Pi.
-
Switch the SSD to the USB 2.0 bus and see if that stays stable long term. That’s the last test I can think to run. Of course, if that works, there’s a good argument for just going back to a large SD card if I want to keep the 2.5GbE adapter connected.
:
