I have not been able to get a RPi4 with the most current eeprom to make a tftp request to a dnsmasq server. It acquires a dhcp address but never sends a tftp request. Is there some trick to this? Online docs I have found make this seem automatic. But then all of the examples use Raspberry PI OS. My dnsmasq server is a Manjaro x86_64 VM.
It seems to work if you add the tftp server IP address in the bootloader config.
While adding the TFTP_IP to the bootloader config worked… if I then remove the setting and reboot, it continues to work. 
Edit: I later used a different RPi4 and it worked without issue.
This seemingly simple config and setup has been rather frustrating for one reason or another. I now have an issue with the RPi4 not issuing an NFS request for the rootfs. The kernel config looks good, I think it is time to take a peek at the initramfs.
Hmm, think I found the issue.
It appears that mkinitcpio-nfs-utils is required.
This is very interesting thank you for documenting your research 
You’re following the Using net section from the mkinitcpio wiki page, right ?
Although I was a bit confused when you mentioned bootloader, I though it was possible to tftp boot directly with the raspberry pi firmware.
I have my pi also get network on boot but to start tinysshd so I can unlock its encrypted root filesystem over the local network.
This is still a work in progress for me. I have made some progress in integrating into our production environment. I now have the RPi4 talking to our production dhcpd server and accessing tftp and NFS from a FreeNAS test VM.
I don’t think an encrypted root filesystem is in my plan, but my plans change as this project progresses.
I will next try to make as much of the root fs read only as I can, but I am not sure if this be truly helpful in the end. I will keep posting here as this part of my project progresses.
Maybe you should look into overlayfs which is used by livecds.
You use a read only nfs share as underlying read only base tree and the upper layer is written either to tmpfs or a local backing drive.
The nfs share could even be a mounted squashfs file which makes definitely sure it is read only.
Excellent suggestion, thank you. I will look into this idea, it might be just what I am looking for.
@Strit Maybe you can guide me. I am looking into using overlayfs on boot, it appears the systemd-volatile-root.service is not included in the initramfs after switching udev to systemd in the HOOKS.
Do I add the missing systemd service file and binary in the mkinitcpio.conf? Or maybe I am missing some additional mkinitcpio hooks package?
Edit: I think this bug was addressed here
Attempting to examine the error is a bit tough since I can not get to a prompt.
However, if I attempt to run the unit file (which is probably not valid after booting) I see the following:
$ systemctl start /usr/lib/systemd.systemd-volatile-root
Failed to start usr-lib-systemd-systemd\x2dvolatile\x2droot.mount: Unit usr-lib-systemd-systemd\x2dvolatile\x2droot.mount not found
This file does not exist on my system.
Edit: This seems to be irrelevant, ignore it.
This works!
MODULES=(overlay)
Edit: Now to backtrack and undo a few things that I tried to find what is required.
This is what I think is needed to get systemd to do the overlay work:
Install:
$ pacman -Syu mkinitcpio-systemd-tool
Then edit /etc/mkinitcpio.conf and modify your settings to include:
MODULES=(overlay)
BINARIES=(/usr/lib/systemd/systemd-volatile-root)
FILES=(/etc/os-release /usr/lib/os-release /usr/lib/systemd/system/systemd-volatile-root.service)
Edit HOOKS, by removing “udev” and replacing it with “systemd”:
HOOKS=(base systemd autodetect modconf block filesystems keyboard fsck)
Then change directory to /boot and run:
$ mkinitcpio -P
Finally modify your /boot/cmdline.txt by adding:
systemd.volatile=overlay
Now reboot and you should see something like the following for the root filesystem:
$ df
Filesystem 1K-blocks Used Available Use% Mounted on
overlay 925600 71388 854212 8% /
I am currently unsure if changing “rw” to “ro” in the cmdline.txt file makes any difference.
Warning: Play with this on a new SD image. Easy to get stuck in a spot with a broken image.
3 Likes
Appreciate you posting the results of your testing here for others to use as a guide.
1 Like
For those of you keeping score at home… I am now attempting to combine the nfsroot and overlayfs… however, it seems there is an issue with systemd performing switch-root to an nfs mount. Working on it.
@Strit @Darksky Any thoughts on when systemd-247 will land in arm-stable?
Edit: Eh, never mind. I will just move forward with arm-testing.
Well, after spending far too much time and effort to get systemd to work with nfsroot, much to my surprise, it simply does not support it. Oh well, I learned stuff in the attempt.
I used pamac-manager to install raspberry-overlayroot-git from the AUR.
Reverted the systemd/udev change in /etc/mkinitcpio.conf and added overlayroot to HOOKS= and to the /boot/cmdline.txt file.
Rebuilt the initramfs and rebooted…
And it works, overlayfs on top of a read only nfsroot.
1 Like
Note: systemd really ought to be able to boot via nfs, but it must still be bugged. This has been buggy in the past and I assume it is once again.
I believe what currently happens is the Switch Root fails, presumably due to a bug in either systemd itself, or the systemd HOOK. It would seem systemd fails to mount the nfsroot as /sysroot.
See this for a “fix” for an issue with HOOKS=(net) which is required for nfsroot and the “missing” wifi device.