Enabling pcie-gen2 on rockpi4(c)

ARM Technical Assistance
1

I am now the lucky booter of Manjaro ARM directly from nvme on my rockpi4c! I owe this advance to the rockpi4b armbian installer which installs the armbian u-boot which is also perfectly happy booting Manjaro ARM for the rockpi4c, If you have a rockpi4 b or c with a SPI chip

I’d recommend rockpi4 owners install Pak0sts kernel to get 4K video output working until it appears in mainline:

If I run:

hdparm -t /dev/nvme0n1

Under Manjaro, it only gives me read speeds of ~200 MB/s with my Samsung EVO 960 which should easily manage 4x that (on a PC), if not more.

If I install the radxa Ubuntu, I can edit /boot/hw_intfc.conf and uncomment the line:

intfc:dtoverlay=pcie-gen2

As documented here: Rockpi4/install/NVME - Radxa Wiki

After doing this and rebooting I get in excess of 400 MB/s so I’d like to replicate this tweak under Manjaro.

So far I have tried creating that file with that line in and also adding dtoverlay=pcie-gen2 as a kernel boot parameter (in /boot/extlinux/extlinux.conf) but neither has increased my SSD speed so there are two likely possibilities left. Could this be a kernel feature or module that I’ll need to rebuild the kernel for (if the relevant patch/feature exists for mainline? I’ve not attempted to rebuild Manjaro ARM yet) or I have got to do this via a u-boot command?

Hopefully someone has beat me to this!

2

The kernel parameters to force PCIe gen 2 disk speed are supposedly libata.force=5.0G and gen1 libata.force=2.5G according to this thread:

https://forum.pine64.org/showthread.php?tid=13931

I’ve tried booting using libata.force=5.0G but I still only get 200 MB/s so it makes me suspect my kernel is lacking the correct pcie-gen2 support.

3

This is Armbians patch for adding pcie-gen2 support on rk3399. I’ve not tried building a Manjaro kernel with it yet.

1 Like
4
5

I had a go at rebuilding pak0sts Manjaro kernel with this patch added unmodified last night but it failed at the linking stage. I think I’ll have to strip it down so it is just the pci-gen2 dtbo instead of including the other bits.

6

Hello. It would be interesting to be able to take advantage of this Rockpi4 feature. I’ve tried it on Armbian, and Debian, and never got encouraging results. There was no significant difference.

7

Under both Radxa Ubuntu with pcigen2 overlay and under FreeBSD I get about 200 MB/s read speed from the nvme, roughly twice the speed I get under current Manjaro kernels without PCIe-gen2.

I need to give NetBSD another go soon. It might be possible to boot NetBSD using the RockPro64 uboot. Ive heard PCIgen2 is working under netbsd for rockpro64 users.

8

I have got NetBSD booting on the RockPi4 but the nvme controller isn’t supported yet.

9

Up until today I had only tried the pcie-gen2 overlay on Radxa Ubuntu but today I verified that it works under Armbian too.

To activate it under Armbian, you need to edit /boot/armbianEnv.txt and add the line:

overlays=pcie-gen2

Then reboot. Without this line, I get ~200 MB/s read speeds off my Samsung EVO 960. With the pci-gen2 overlay enabled I get ~360 MB/s which is definitely a notable and worthwhile improvement.

I had a couple of unsuccessful, super-hacky attempts to add this to the Manjaro kernel and intend to give it another go soon.

10

Add the same overlay to Manjaro and hopefully it should work.
You will need dtbo file and add it in extlinux.conf to get it loaded on boot.

1 Like
11

I tried using the pcie-gen2.dtbo from the Armbian (5.13.12) kernel with the Manjaro (5.17.1) kernel but it failed to boot so it looks like the dtbo will need to be built specifically for the Manjaro kernel.

The syntax for loading a dtbo with extlinux under Manjaro is different from how they are loaded under Armbian. It seems the correct way to load a dtbo under Manjaro is to add a line like:

FDTOVERLAYS /path/to/pcie-gen2.dtbo

After the FDT line in extlinux.conf.

This page states:

“It (DTB overlays) requires both the base and all the overlays to be compiled with the -@ command line switch so that symbol information is included.” which I expect won’t be the case with the current Manjaro rockpi4 kernel?

Here’s the post explaining the fdtoverlays keyword:

https://lists.denx.de/pipermail/u-boot/2021-January/436577.html

12

It actually is compiled with it, for this reason.

What you might be missing is that the u-boot needs to support overlays too, which it currently does not for the Rock Pi 4. You need to add CONFIG_OF_LIBFDT_OVERLAY=y to the config used to build u-boot with for that.

1 Like
13

Thanks Strit

Could we get that build option enabled by default for future u-boot builds?

I should be able to rebuild u-boot and try this again tonight.

14

I think we can enable it in our next uboot builds.

In talk with a developer, he says that he would not recommend enabling pcie 2.0 on an rk3399 board. He wanted to let you know that:

Also he would like you to do the read speed test using these commands instead:

dd if=/dev/nvme0n1 of=/dev/null bs=1M count=2048 iflag=direct status=progress
dd if=/dev/nvme0n1 of=/dev/null bs=4M count=512 iflag=direct status=progress
15

Yesterday I rebuilt the rockpi4c u-boot package via:

I modified the PKGBUILD to enable CONFIG_OF_LIBFDT_OVERLAY=y but, after installing the modified u-boot it still failed to boot when loading the dtbo from Armbian.

Yes, I was aware of some people claiming to have probs with the gen2 overlay, hence why we should keep at as an optional overlay. It would seem to depend on your make and model of SSD what kind of results you get.

I will get those dd benchmarks and hdparm benchmarks for my SSD under both Manjaro and Radxa Ubuntu, the latter with and without the pcie-gen2.dtbo over the weekend. I seem to get better transfer rates with the Radxa Ubuntu image vs Armbian or Manjaro.

16

I just updated the uboot packages in unstable branch to have the overlay support enabled. So hopefully it should work if you get a working DTS for the kernel.

1 Like
17

It doesn’t seem I can edit my post above. The Armbian rk3399 pcie-gen2 patch has been moved to:

18

At last, I have a fully functional Rock Pi 4c with PCIe 2.0 NVME, 4K display support, ethernet, wifi and bluetooth support!

I have patched the rockpi 4b dtb that is included with pak0st’s 4K kernel. I actually have a 4c but Manjaro currently detects it as a 4b. Without the patched dtb file, I get about 201 MB/s read from my Samsung EVO 960 NVME SSD. After patching the DTB I get about 340 MB/s.

HOWTO patch your dtb for PCIe 2.0 support

First install dtc to decompile the dtb:

pacman -S dtc

Decompile the dtb to a dts:

dtc -I dtb -O dts -o /boot/dtbs/rockchip/rk3399-rock-pi-4b.dts /boot/dtbs/rockchip/rk3399-rock-pi-4b.dtb

Edit /boot/dtbs/rockchip/rk3399-rock-pi-4b.dts and change the pcie device max-link-speed to <0x02>, probably on line 245, then:

mv /boot/dtbs/rockchip/rk3399-rock-pi-4b.dtb /boot/dtbs/rockchip/rk3399-rock-pi-4b.dtb.old

Build the patched dts:

dtc -O dtb -o /boot/dtbs/rockchip/rk3399-rock-pi-4b.dtb /boot/dtbs/rockchip/rk3399-rock-pi-4b.dts

Reboot and PCIe 2.0 should be enabled.

19

If this fix is safe to use, maybe you could upstream it?

1 Like
20

Yes. I’ve only just done this now so let me test it a bit first.

I’m going to compare building some software an old i5 I have with 4 GB RAM running Arch (EndeavourOS) to my RockPi 4 running Manjaro first.

1 Like