uCM4 was designed as a network device. It has Ethernet connector, which is the most important interface to outer world. And these days many devices, including our PCs can boot directly from network using thing called PXE. For uCM4 PXE would make Micro SD connector unnecessary. Luckily for us, for quite some time Raspberrys also can boot themselves from network. CM4 is not an exception here. But unlike ordinary Raspberry Pi 4, it is a bit more complicated. Let me present how to achieve that with CM4 and uCM4. I would not be surprised, if you got here while looking for a way to do that with CM4 and official CM4IO, so I will try to mention the differences between uCM4 and CM4IO in that matter. Enough of this introduction, let's start!

What you need

Basically, there are few Git repos to clone to allow you to get started. Clone them by issuing following into your system (this does not necessarily has to be Raspberry and keep in mind that for uCM4 you will have to be able to power whole system from that device, so my advice is to use PC, or alternatively device with active USB hub in the middle):

git clone https://github.com/raspberrypi/rpi-eeprom.git
git clone https://github.com/raspberrypi/usbboot.git

If using uCM4, you also gonna need tweezers, or something similar to short JP1 jumper. I know, this is one of the not-so-successful parts of the design and is going to be subject to change in future.

What's more, on the system that will serve as server for Raspberry to communicate with you have to install DHCP, TFTP and NFS servers. In case of Arch Linux these can be provided by dnsmasq (DHCP+TFTP) and nfs-server (NFS). In other distros names should be similar.

Getting software ready

Fortunately programs in rpi-eeprom repo are in fact scripts, so nothing to do there. But in usbboot, you actually have to compile. Luckily this is as simple as calling:

make

As a result you should get rpiboot binary directly in repository root directory.

Getting right EEPROM firmware image

This is harder part. From what I know, it is not possible to simply change configuration in EEPROM. You have to flash full firmware image with changed configuration. What I did was to simply copy recovery firmware from usbboot repo, but in fact you can use any version from rpi-eeprom repo. The one that I chose is pieeprom-2020-10-02.bin, so this is how I will refer to it later.

Preparing configuration

As you have your pieeprom image, you can go to usbboot repo and create new directory for our network configuation. Copy your EEPROM image to this new directory. Don't change the name for now, we will make use of this improper name later. You will also need bootcode4.bin from recovery as first stage bootloader, I guess. Now it would be nice to get current boot.conf from CM4 that we are going to reconfigure. While in Raspberry Pi OS, you can simply call:

vcgencmd bootloader_config

And save the output as boot.conf of your PC. You should get something like me:

[all]
BOOT_UART=0
WAKE_ON_GPIO=0
POWER_OFF_ON_HALT=1
DISABLE_HDMI=0
BOOT_ORDER=0xf41

At this point the only change for us to do is to replace 0xf41 with 0xf21. Why? Here is the documentation from Raspberry Pi Foundation. Basically this 4 meant USB-MSD and 2 means NETWORK, so we create order like SD->NETWORK->RESTART.

Updating EEPROM image with new config

We have our configuration, but not in a file used by rpiboot. So, we have to insert it, yet. And this is why we really needed rpi-eeprom repo. First, we need to add it to our PATH:

export PATH="$PATH:/path/to/rpi-eeprom"

Then we have to call rpi-eeprom-config to modify pieeprom.bin:

rpi-eeprom-config --config network/boot.conf --out network/pieeprom.bin network/pieeprom-2020-10-02.bin

This creates the file that rpiboot expects - pieeprom.bin. Finally, we need to create signature for our image. Let's start by copying template from recovery directory. Then call sha256sum to see current hash:

cp recovery/pieeprom.sig network/
sha256sum network/pieeprom.bin

Replace hash in the first line of sig file with what you got, so it looks more or less like below:

384a24cbb35d648de76fb19f5bef51b895e4a3d35ef0692f85045e5e6a6ec238
ts: 1628266219

That's it. We are ready to go!

USBbooting uCM4

Now you have to get your CM4 into the state, where it is connected to power, but turned off. In theory it is possible to short JP1, while connecting USB, but it is way easier to short it, then while keeping it short, simply push POWER button. You should immediately see that activity LED is not blinking which means that most likely it succeeded. We can verify that we indeed shorted it in the right moment by checking lsusb on PC. One new device should appear:

Bus 001 Device 017: ID 0a5c:2711 Broadcom Corp. BCM2711 Boot

This means that we have BCM2711 in mode called rpiboot. This allows to boot it with rpiboot tool via USB connection to our PC.

In this step users of CM4IO board are more lucky, as CM4IO is development board, so there is goldpin jumper for that and it is clearly described on silkscreen:

So, if you are lucky to have CM4IO, you just put a jumper to first position of J2 connector, connect to power and should be ready for flashing.

Flashing

Now what is left is just flashing EEPROM:

sudo ./rpiboot -d network

Following output should appear on Linux console:

Loading: network/bootcode4.bin
Waiting for BCM2835/6/7/2711...
Loading: network/bootcode4.bin
Sending bootcode.bin
Successful read 4 bytes 
Waiting for BCM2835/6/7/2711...
Loading: network/bootcode4.bin
Second stage boot server
Loading: network/pieeprom.bin
Loading: network/pieeprom.bin
Loading: network/pieeprom.sig
File read: pieeprom.sig
Loading: network/pieeprom.bin
File read: pieeprom.bin
Second stage boot server done

And this is what you should see on UART:

SIG pieeprom.sig 384a24cbb35d648de76fb19f5bef51b895e4a3d35ef0692f85045e5e6a6ec238 1628266219
Read pieeprom.bin bytes   524288 hnd 0x00000000 sha256 384a24cbb35d648d              
Reading EEPROM: 524288                                                                   
Writing EEPROM                                                                      
.................................................................................+..............................................
Verify BOOT EEPROM                                                                                   
Reading EEPROM: 524288                                                                                        
BOOT-EEPROM: UPDATED

This means that we are fine!

Verifying

Previous step should leave you in blinking LED state, so you have to reset CM4. Do not try PXE, yet and leave SD card on its place.

Before actually trying netboot, let's verify that we have network boot enabled by calling again:

vcgencmd bootloader_config

 As a result, you should now see:

[all]                                                  
BOOT_UART=0                                                               
WAKE_ON_GPIO=0                                                                    
POWER_OFF_ON_HALT=1                                                                        
DISABLE_HDMI=0                                                                                 
BOOT_ORDER=0xf21

Netbooting

Now, we can go standard path, as with any other device that we want to netboot. For this you could try to follow official tutorial for Raspberry Pi, but my opinion is that it is overengineered, if the only thing we want to do is to test things out. There are only two important configuration points to make it work. First is to setup DHCP and TFTP server together, as both are delivered by dnsmasq. Second is to start NFS server to share root filesystem.

DHCP+TFTP

First thing that Raspberry Pi will try after bootup is getting IP address. For this you can obviously go official way and set connection up on other Raspberry, or you can make it simpler. My idea is to use ordinary PC as a server. Other option that I can think of is to use home router. But for this, that can't be an ordinary, cheapest router that you could get on ebay. It won't work that way. Mikrotik for sure could do that as well, if only you can connect USB stick to it. Anyway, getting back to my setup. What I want to do is to unplug Micro SD card from uCM4, plug it into PC and use it directly.

For this, we need Ethernet connection active with static IP address. NetworkManager could do that, netctl too, so I won't describe that here.

I assume that I have boot partition of my Raspberry at /mnt/boot and rootfs at /mnt/rootfs. There is one small problem in using boot directly. We need to modify cmdline.txt, so it will use NFS, instead of rootfs partition. If we do that directly, we would have to use NFS even if we plug SD card back into uCM4. So it is better idea to mirror boot partition, let's say into /tftpboot:

mkdir /tftpboot
cp /mnt/boot/* /tftpboot/

For now cmdline.txt could stay unmodified, it will be needed when we have NFS share.

Now create dnsmasq.conf file and put something like that inside

port=0
dhcp-range=192.168.2.100,192.168.2.254
log-dhcp
enable-tftp
tftp-root=/tftpboot
pxe-service=0,"Raspberry Pi Boot"

It is important to have dhcp-range within range defined on your Ethernet connection. Otherwise, it fails.

Now the server could be started with:

dnsmasq -dC dnsmasq.conf -i eth0

 Provided that you are in directory, where your dnsmasq.conf is and you have root privileges.

And basically this is it. At this point you should be able to boot your kernel by simply powering CM4 on. In dnsmasq logs, you should see TFTP requests for Raspberry Pi boot files. Obviously this will stop at mounting rootfs:

[    3.058370] Waiting for root device PARTUUID=12345678-01...

cmdline.txt

Now it is time to adjust kernel arguments to use NFS instead of SD card. By default it is set like:

console=serial0,115200 console=tty1 root=PARTUUID=01234567-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait

We need to change to something like:

console=serial0,115200 console=tty1 ip=dhcp root=/dev/nfs nfsroot=192.168.2.1:/mnt/rootfs,vers=4.1,proto=tcp elevator=deadline fsck.repair=yes rootwait 

Don't forget to adjust IP address and location of rootfs!

NFS

Last thing is to set NFS up. This is as easy as modifying /etc/exports on your server and adding export like this:

/mnt/rootfs *(rw,sync,no_subtree_check,no_root_squash)

Then you have to reload those changes into NFS server with:

sudo exportfs -arv

 And start NFS server. On Arch it is like that:

systemctl start nfs-server

But could be slightly different on other distros, so refer to their documentation, especially if your distro did not migrate to systemd!

Last thing that will prevent your Raspberry from booting is /etc/fstab, which still will attempt to mount SD card and will fail after timeout period. My workaround, if you want to use exactly same image for both PXE and SD boot is to move fstab file to /boot and create symlink from /boot/fstab to /etc/fstab. This way we could have two different fstab files - one on tftpboot, one on SD card's boot partition. This could be done with:

cp /mnt/rootfs/etc/fstab /mnt/boot/
cp /mnt/rootfs/etc/fstab /tftpboot/
ln -sf /boot/fstab /mnt/rootfs/etc/fstab

 Finally, we can change just /tftpboot/fstab and comment out SD-related entries:

proc            /proc           proc    defaults          0       0
#PARTUUID=12345678-01  /boot           vfat    defaults          0       2
#PARTUUID=12345678-02  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that

At this point, we can attempt booting CM4 with Ethernet connected, all servers running and connection set up on server side.
Actually this makes fstab to be unreachable in Raspberry Pi OS, when netbooted, but despite that, it works. For proof of concept it is enough. For the final solution, not really. But for final setups, I can imagine that one wants to have separate images for netboot and SD boot anyway. Having it common is only an advantage for testing.

Now, when powered your CM4 should start from network and you should clearly see in logs of dnsmasq, CM4 UART, or Wireshark that it did. That's it!