Welcome to the party. It’s essentially like trying to get linux to run on proprietary laptop hardware in the early 2000s. There’s a few that work 100%, and the rest are kinda a work in progress. I’ve just recently broken into arm and I haven’t really found any good guides anywhere to the differences, but I’d say it’s essentially 80% the same, as you said. The differences aren’t really as pronounced as you might think, but you’re right, HAL is a key one.
The thing that is most striking is the boot process and the concept of overlays.
https://source.android.com/devices/architecture/dto
Imagine if you will that the hardware on your device is not plug and play and you can’t just auto detect things on the pci or usb bus and go “oh, that’s device aaaa:bbbb, lets go search for support”. While I’m no expert, essentially you need, as part of your boot process, access to the right overlay files (and any config files to tweak their settings for your specific boot loader) to complete the device tree that will be available for the kernel once loaded to know what devices are available and need drivers.
That explanation precludes the existence of the bootloader, which is usually provided by the device manufacturer, but there are other ways to boot the device such as projects that provide UEFI style bios similar to x86. Every device is a little different, so I’ll describe the process on my Raspberry Pi 4b (which honestly is one, if not the best, supported devices along with the Pinebook Pro.)
The way the pi works is the SoC (gpu, or cpu, honestly, depending on board version) will consult its eeprom for the lowest level possible configuration variables. On the PI, this includes the boot order and any other low level things like USB tweaks, tftp settings, etc. It then uses the boot order much like an x86 UEFI filesystem search to find it’s boot code on some fat filesystem available in the (I believe) the first partition of each device in the boot order eeprom var. This “EFI like” partition is actually referred to as the firmware partition or boot partition and contains the device tree overlay files, the boot loader elf entry point files, the boot loader configuration, the linux kernel, and the linux initrd. For the pi, there are two main config files: config.txt that configures the hardware and overlays and cmdline.txt which configures the arguments to the linux kernel. Indeed, the RPI boot loader is literally a “linux kernel starter” program, as the Raspberry Pi Foundation designed the device to boot linux. The boot loader takes the place of grub. You can think of it as “the bios and grub combined”. Other devices are definitely different, and there are various projects that are the “defacto sandard” way to boot linux on those devices, but us Pi Peeps are definitely lucky because the RPI foundation decided linux was their thing and so it’s comparatively a snap to support.
On other devices, which I will admit I have no experience on, but am actively gaining as I consider buying a Odroid N2+ and others, follow a similar trend, especially when it comes to overlays, but there are differences in boot procedure.
One thing that seems to be quite common is that instead of using X86 style “removable media installer that installs onto target persistent storage,” you’ll find a trend that installers will generate images or you download pre-built images of your “installed os” that will configure itself on first boot. The theme among these is as follows:
- Boot up
- Add non root user with sudo capabilities and right user groups
- resize filesystem to max size
- start user environment.
It would be up to you to partition another device such as emmc or a usb ssd and transfer the resulting set of LABELED PARTITIONS or UUID identified partitions to another device (including resizing) and setup the boot loader to find and use them. This is how MJ on RPI worked, and honestly every other linux I used on it. Devices like pinebook pro and others that have built in EMMC module support may offer the ability to deploy on a target device like that, but it’s hard for me to speak to it with no experience outside my Pi.
On Pi, one of the biggest lessons is to get a good SD CARD as it can make or break your environment. There are numerous benchmarks available (and I’ve done a lot myself) and the key take away is this: Samsung EVO Plus or SELECT is incredible, and SanDisk Extreme are pretty good too, just don’t buy into that A2/A1 rating nonsense as it requires a different device/driver setup that is pointless at this point, and some of the A2 cards are actually slower than their A1 counterparts in our card readers. Also, don’t assume that the USB SSD you’re thinking of getting will just work, devices can be far more picky than you’d think in arm land.
As for software, ARM, while the platform is decidedly “more closed” than x86 is now days, it will shine a bright light on what software and services are closed too, such as video providers that want DRM, or programs that are closed source and you simply cannot run as there is no ARM binary (at least not arm linux!). The quintessential examples of this are widevine, discord, and pandora in my case.
Perhaps the most important thing to realize is that while the Apple M1 chip is x86 fast, none of the ones you will have linux support on today are. Just prepare yourself. Linux on arm has historically been for embedded applications and not desktop applications. Due to the closed nature of arm device manufacturers, with some clear exceptions, you are years behind what just works with a vendor supplied android load. This translates into the following truth at the beginning of 2021: Your arm linux experience will be decidedly slower in all aspects than anything x86. It’s currently a labor of early adopter love, although arm linux has been around for a long time, the new RPI 4b, pinebook pro, and apple M1 just, in my oponion, just injected some steroids right into the whole linux on arm concept. Apple showed us the way, the RPI 400 “arm computer in a keyboard” exists, the pinebook pro exists, and there are a bunch of existing cromebook and windows arm notebooks and it’s clear that this is the way forward and the market will never be the same thanks to Apple’s actions.
2020 and previous: So you wanna do some embedded apps?
2021 on onward: Gimme supported desktop now!
I literally have numbered sd cards all over my desk from all the different OS loads I’ve been trying 
The RPI isn’t going to be my only arm device. I’m here for the PANICs
Wayne
