Building a minimal AArch64 root filesystem for network booting

In many cases we may want to quickly boot a minimal root filesystem on our SBC:

  • To quickly test our latest, just compiled version of u-boot or of the Linux kernel.
  • To test our network boot (PXE) setup.
  • For other development purposes where we don't want to have to deal with a full-fledged Linux distribution such as Debian or Ubuntu.

This is a quick guide / HOWTO first, build a minimal AArch64 root filesystem and second, test it using QEMU - including booting it from the network, just as we would do with a real SBC.

Our minimal root filesystem will include a statically-linked BusyBox and the Dropbear SSH server, and yet it will still fit with room to spare in an 8MB file!

Basic steps

  • Install Ubuntu 18.04 on your laptop, PC or workstation, and customize to taste (note that the following instructions should work fine with any other recent Linux distribution, with very few changes).
  • Create a new directory under your home directory and call it for example "Amlogic". Alternatively, create a partition with label Amlogic and mount it (it will be mounted under /media/<username>/Amlogic), then change the group for the mount point to adm and set group write permissions.
  • Inside <path>/Amlogic, create a folder min-rootfs_dev.
  • Install the cross-compilation toolchain, git and a few more tools:
> sudo apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu
> sudo apt-get install build-essential autoconf libtool cmake pkg-config git python-dev swig3.0 libpcre3-dev nodejs-dev gawk wget diffstat bison flex
> sudo apt-get install device-tree-compiler libncurses5-dev

Getting Busybox and Dropbear

We are going to download the latest versions of all software packages from their respective websites directly into our <path>/Amlogic/min-rootfs_dev directory and decompress them.

> cd .../Amlogic/min-rootfs_dev/
> wget https://busybox.net/downloads/busybox-1.29.3.tar.bz2
> tar -xjf busybox-1.29.3.tar.bz2
> wget https://matt.ucc.asn.au/dropbear/releases/dropbear-2018.76.tar.bz2
> tar -xjf dropbear-2018.76.tar.bz2

Compiling a statically-linked BusyBox

This is really simple and took less than 3 minutes on my old and trusty Thinkpad T510.

> cd .../Amlogic/min-rootfs_dev/busybox-1.29.3/
> make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- defconfig
> make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- menuconfig

Remember to set the option for a statically-linked BusyBox (in Settings - Build options):

busybox1.png

Save the configuration and exit menuconfig. Now we can cross-compile BusyBox:

> time make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-

Compiling Dropbear static

This is also simple and took just under 1 minute on my old and trusty Thinkpad T510.

> cd .../Amlogic/min-rootfs_dev/dropbear-2018.76/
> ./configure --host=aarch64-linux-gnu --prefix=/ --disable-zlib --enable-static CC=aarch64-linux-gnu-gcc LD=aarch64-linux-gnu-ld
> time make

Building the root filesystem

Now it's time to create our root filesystem, install the programs we just compiled, create all the required directories, and adjust some more details.

First we create the rootfs directory:

> cd .../Amlogic/min-rootfs_dev/
> mkdir rootfs

Now we install BusyBox:

> cd .../Amlogic/min-rootfs_dev/busybox-1.29.3/
> make ARCH=aarch64 CROSS_COMPILE=aarch64-linux-gnu- install CONFIG_PREFIX=../rootfs
> cd ..
> ls ./rootfs/

You'll notice BusyBox already created a few directories for us. Let's install Dropbear:

> cd .../Amlogic/min-rootfs_dev/dropbear-2018.76/
> make install DESTDIR=../rootfs
> cd ..
> ls ./rootfs/sbin/

And you'll notice Dropbear is installed in the sbin directory, as expected. We'll generate the keys when we test our root filesystem in QEMU later. For now, let's wrap things up by taking care of a few more details.

> cd .../Amlogic/min-rootfs_dev/rootfs/
> mkdir proc sys dev etc etc/init.d usr/lib home home/root root

Finally, create an empty file etc/init.d/rcS, make it executable:

> touch etc/init.d/rcS
> chmod +x etc/init.d/rcS

And edit it with your favorite text editor so that it contains the following shell script:

#!bin/sh
mount -t proc none /proc
mount -t sysfs none /sys
echo /sbin/mdev > /proc/sys/kernel/hotplug
/sbin/mdev -s

Cross-compiling a test kernel

We are ready to test our minimal Aarch64 root filesystem using QEMU! Well, almost ready anyways… because first we need a kernel!
We are going to use this custom kernel configuration file to cross-compile an Aarch64 kernel following the instructions in this Wiki page: HOWTO cross-compile the Linux kernel for the Libre Computer AML-S805X "La Frite"
Just load the custom configuration file in the make menuconfig step and then copy the compiled kernel Image file found in …/linux/arch/arm64/boot/ to our …/Amlogic/min-rootfs_dev/ directory

Testing the root filesystem using QEMU

As a final step we have to generate a small raw image of an ext3 filesystem with our minimal root filesystem in …/Amlogic/min-rootfs_dev/:

> cd .../Amlogic/min-rootfs_dev/
> dd if=/dev/zero of=minrootfs.ext3 bs=1M count=32
> mkfs.ext3 minrootfs.ext3
> mkdir -p tmpfs
> sudo mount -t ext3 minrootfs.ext3 tmpfs/ -o loop
> sudo cp -r rootfs/* tmpfs/
> sudo sync
> sudo umount tmpfs
> rmdir tmpfs

And we can finally launch QEMU with the following command:

qemu-system-aarch64 -smp 2 -M virt -m 1024 -cpu cortex-a53  -kernel Image -append 'root=/dev/vda console=ttyAMA0' -drive if=none,file=minrootfs.ext3,id=hd0 -device virtio-blk-device,drive=hd0  -netdev user,id=mynet -device virtio-net-pci,netdev=mynet -nographic

References

  1. Building a minimal RootFS with Busybox, GLIBC and DropBear.
  2. Build Your ARM Image for QEMU.
  3. Build a minimalist root filesystem.
  4. Build and boot a minimal Linux system with qemu.