iPXE Boot over HTTP: Kernel Errors

[Windows7ge]

For the fun of the adventure I’m exploring iPXE OS Installation functionality across various Linux distributions and documenting my discoveries with the more popular distros. I just recently found success with Arch Linux but Manjaro is proving to be more complicated. Unlike the issues I had with Arch Manjaro’s Kernel isn’t telling me what file it’s looking for. It just starts trying to perform mount operations then fails.

:: running early hook [udev]
Starting version 251.2-3manjaro
:: running early hook [miso_pxe_nbd]
:: running hook [udev]
:: Triggering uevents...
:: running hook [miso]
:: running hook [miso_loop_mnt]
:: running hook [miso_pxe_common]
IP-Config: eth0 hardware address 66:90:71:1d:d3:a6 mtu 1500 DHCP RARP
IP-Config: eth0 guessed broadcast address 10.0.0.255
IP-Config:  complete (from 10.0.0.15):
 address: 10.0.0.32        broadcast: 10.0.0.255       netmask: 255.255.255.0
 gateway: 0.0.0.0          dns0     : 0.0.0.0          dns1   : 0.0.0.0
 domain : example.org
 rootserver: 10.0.0.15 rootpath:
 filename  : ipxe.efi
:: running hook [miso_pxe_http]
:: running hook [miso_pxe_nbd]
:: running hook [miso_pxe_nfs]
:: running hook [keymap]
:: Loading keymap...done.
:: Mounting /run/miso/httpspace (tmpfs) filesystem, size='75%'
:: Mounting /run/miso/copytoram (tmpfs) filesystem, size=75%
:: Mounting /run/miso/cowspace (tmpfs) filesystem, size=256MB...
:: Mounting overlay root (tmpfs) filesystem, size=75%...[    9.287676] overlayfs: unrecognized mount option "lowerdir=" or missing value
mount: /new_root: wrong fs type, bad option, bad superblock on overlay, missing codepage or helper program, or other error.
       dmesg(1) may have more information after failing mount system call.
:: running late hook [miso_pxe_common]
:: running cleanup hook [miso_shutdown]
cp: cannot create directory '/run/initramfs/': No such file or directory
:: running cleanup hook [udev]
ERROR: Failed to mount the real root device.
Bailing out, you are on your own. Good luck.

sh: can't access tty; job control turned off
[rootfs ]# _

Serving as the iPXE server I have Ubuntu Server running DHCP/TFTP/HTTP and a segment of the iPXE boot menu is as follows:

:arch
kernel ${boot-url}${file-path5}/boot/x86_64/vmlinuz-linux
initrd ${boot-url}${file-path5}/boot/x86_64/amd-ucode.img
initrd ${boot-url}${file-path5}/boot/x86_64/intel-ucode.img
initrd ${boot-url}${file-path5}/boot/x86_64/initramfs-linux.img
imgargs vmlinuz-linux initrd=amd-ucode.img initrd=intel-ucode.img initrd=initramfs-linux.img archiso_http_srv=${boot-url}/installs/linux/ ip=dhcp
boot

:manjaro
kernel ${boot-url}${file-path6}/boot/vmlinuz-x86_64
initrd ${boot-url}${file-path6}/boot/amd_ucode.img
initrd ${boot-url}${file-path6}/boot/intel_ucode.img
initrd ${boot-url}${file-path6}/boot/initramfs-x86_64.img
imgargs vmlinuz-x86_64 initrd=amd_ucode.img initrd=intel_ucode.img initrd=initramfs-x86_64.img miso_http_srv=${boot-url}/installs/linux/ ip=dhcp misobasedir=manjaro misolabel=manjaro
boot

I was hoping to find success basing what little information I can find by using arch as a point of reference but some resources are saying that Manjaro is not a distro that can be installed over HTTP. I’m only seeing small resources showing success over NFS.

Input from anyone with better insight is appreciated.

[Windows7ge]

Figured out the problem. Like Arch Manjaro wants to do the same thing but doesn’t tell you it on screen.

Insides the .ISO file everything inside the folder /manjaro/x86_64/ has to be copied to the HTTP/TFTP server. Additionally once added to the server those files HAVE to go inside /manjaro/x86_64/.

Then miso_http_srv= has to point to the parent directory of manjaro.

With this in mind during the kernel boot process it will reach out to HTTP/TFTP and look for the folder named manjaro, then it will look inside for x86_64, then it will look for the 8 files inside.

For reference this is my updated config:

#!pxe
set url    http://10.0.0.15/
set iso    ${url}iso/

:manjaro
kernel ${kernel}manjaro/21.3.2/vmlinuz-x86_64
initrd ${ucode}manjaro/21.3.2/amd_ucode.img
initrd ${ucode}manjaro/21.3.2/intel_ucode.img
initrd ${initrd}manjaro/21.3.2/initramfs-x86_64.img
imgargs vmlinuz-x86_64 initrd=amd_ucode.img initrd=intel_ucode.img initrd=initramfs-x86_64.img miso_http_srv=${iso} ip=dhcp
boot

[zbe]

If you don’t mind downloading images over the internet you can also use netboot.xyz. I have atftp running on rpi, and I threw netboot.xyz.efi on it. And then I just have rpi as ‘Next Server’ in my Mikrotik router’s DHCP server configuration.

Manjaro (and some other images) didn’t work for me though.

[Windows7ge]

For the purposes of this application I want it to be a All-In-One self contained appliance. I don’t want it to rely on public repos which may or may not stay online when I want them or perhaps I myself am offline at a time when I want to do an install.

Heck I still have to figure out how to setup ISC_DHCP_SERVER with a Static Route to the home LAN so clients can get Internet as everything I’m reading clains ISC_DHCP_SERVER doesn’t have ProxyDHCP capability. Might have to investigate DNSMASQ as some people have seem to have gotten that working.

And I as well am finding that different distributions have to use different methods of loading the OS as they don’t all support one universal feature to do so. It makes for a great learning experience though because it requires me to do a lot of critical thinking and trial & error. So far this is the list of Linux Installers I have working:

Arch Linux 5.18.7
Bodhi 6.0.0
CentOS 7
ElementaryOS 6.1
Fedora 36 Server (Legacy BIOS Only)
Fedora 36 Workstation (Legacy BIOS Only)
Kubuntu 22.04 LTS
Lubuntu 22.04 LTS
Manjaro 21.3.2
Linux Mint 20.3
PopOS 22.04 LTS
Ubuntu Desktop 22.04 LTS
Ubuntu Server  20.04 LTS
Xubuntu 22.04 LTS

I don’t really have an end goal but if I can cover most of the popular ones and some server and obscure ones I’ll consider this a success.

Right now my biggest trouble is with any distro UNIX based. I can’t mount the .ISO to extract the kernel & initrd. So OS’s like FreeBSD, TrueNAS, etc are off the table.