How to make an ARM virtual machine (armhf/aarch64)

Update (2022-03-19): I wrote about a new way to create an ARM virtual machine that’s simpler and handles kernel updates properly. I highly suggest you follow those instructions instead, unless you are building a chroot.

I noticed that very few people seem to know how to create a full ARM virtual machine, so I decided to create a quick guide.

This tutorial will use aarch64 and Debian as examples, but the same methodology should work for 32-bit ARM and other distributions. The instructions can also be adapted to create a simple chroot.

Step 1: Image Creation

First, we will need to create a disk image. We’ll be using a raw image, which is the most convenient option given you can loop mount. If you are just building a chroot, you can skip this step.

To create an image, we first have to create a file. Just run the following command (change 8G to however big you want the image to be):

$ fallocate -l 8G arm64-vm.img

Then, you should create an ext4 filesystem on the image:

$ sudo mkfs.ext4 arm64-vm.img
mke2fs 1.44.5 (15-Dec-2018)
Discarding device blocks: done
Creating filesystem with 2097152 4k blocks and 524288 inodes
Filesystem UUID: b8455983-417b-4c1c-b170-fd4a4e63c060
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

Record the filesystem UUID. This will be important later.

Then we mount this image somewhere, for example, /mnt/test:

$ mkdir -p /mnt/test
$ sudo mount -o loop arm64-vm.img /mnt/test

Your image is ready to be populated with a chroot! If you are just doing a chroot, instead of /mnt/test, you should replace all instances of /mnt/test with the desired path for the chroot.

Step 2: Setup `qemu` for executables

We will need to install the qemu-arm-static binary. On Debian, simply run:

$ sudo apt install binfmt-support qemu-user-static

Then, we will need to make sure /usr/bin/qemu-arm-static is available inside the chroot:

$ sudo mkdir -p /mnt/test/usr/bin
$ sudo cp /usr/bin/qemu-arm-static /mnt/test/usr/bin

Step 3: Bootstrap the system

We will use debootstrap to populate the chroot. On Debian and similar systems, this can be installed with:

$ sudo apt install debootstrap

Once installed, simply run:

$ sudo debootstrap --arch=arm64 buster /mnt/test/ http://deb.debian.org/debian/

Replace buster with the version of Debian (or Ubuntu) you want to use. Replace http://deb.debian.org/debian/ if you have a better mirror (or want to use Ubuntu). Replace arm64 with armhf if you are building a 32-bit ARM system.

If you want to use another Linux distribution, please follow their guide to create the chroot.

Step 4: Enter the chroot

To enter the chroot, we first need to make critical directories available through bind mounting. To do this, run:

$ for dir in /dev /proc /sys; do sudo mount --bind $dir /mnt/test$dir; done

Then, we can just spawn bash inside the chroot:

$ sudo chroot /mnt/test /bin/bash

If you are doing a simple chroot, this is the end.

Step 5: Prepare chroot for execution as virtual machine

The chroot was built without a kernel, so you will not be able to boot it as a VM with QEMU. So we simply install the kernel:

# apt install linux-image-arm64

If you are building a 32-bit ARM VM, you should use linux-image-armmp instead.

Setting root password

You should also configure a root password so you could login:

# passwd
New password:
Retype new password:
passwd: password updated successfully

Configuring `fstab`

You should configure /etc/fstab by changing it to:

UUID=b8455983-417b-4c1c-b170-fd4a4e63c060 / ext4 errors=remount-ro 0 1

Replace b8455983-417b-4c1c-b170-fd4a4e63c060 with the UUID from step 1.

Configuring networking

You should change /etc/network/interfaces.d/eth0 to:

auto eth0
iface eth0 inet dhcp

Finishing up

Once you are done, exit the chroot.

# exit

Step 6: Prepare for boot

We need to copy the kernel and initrd image out of the VM into some location on the host. For example, /var/lib/libvirt/images:

$ sudo cp /mnt/test/boot/{initrd.img-,vmlinuz-}* /var/lib/libvirt/images

We will then need to unmount the image:

$ for dir in /dev /proc /sys /; do sudo umount /mnt/test$dir; done

Creating the VM with `libvirt`

If you are using libvirt, the following command should create the VM:

$ sudo virt-install \
	--name arm64-vm \
	--arch aarch64 \
	--machine virt \
	--os-type Linux \
	--os-variant debian10 \
	--ram 2048 \
	--vcpus 2 \
	--import \
	--disk /var/lib/libvirt/images/arm64-vm.img,bus=virtio \
	--graphics none \
	--network user,model=virtio \
	--features acpi=off \
	--boot kernel=/var/lib/libvirt/images/vmlinuz-4.19.0-13-arm64,initrd=/var/lib/libvirt/images/initrd.img-4.19.0-13-arm64,kernel_args='console=ttyAMA0,115200 root=/dev/vda net.ifnames=0 biosdevname=0'

If you are doing 32-bit ARM, use this instead:

$ sudo virt-install \
	--name arm-vm \
	--arch armv7l \
	--machine virt \
	--os-type Linux \
	--os-variant debian10 \
	--ram 2048 \
	--vcpus 2 \
	--import \
	--disk /var/lib/libvirt/images/arm-vm.img,bus=virtio \
	--graphics none \
	--network user,model=virtio \
	--boot kernel=/var/lib/libvirt/images/vmlinuz-4.19.0-13-arm64,initrd=/var/lib/libvirt/images/initrd.img-4.19.0-13-arm64,kernel_args='console=ttyAMA0,115200 root=/dev/vda net.ifnames=0 biosdevname=0'

Obviously, replace the --disk path with the path to your image, and similarly for kernel and initrd. Feel free to change --name.

If all goes well, the virtual machine should boot and you should see the serial console. Login as root and run shutdown now.

Your VM should now be created.

Creating the VM with plain QEMU

To run a 64-bit ARM with QEMU directly, do:

$ qemu-system-aarch64 -m 2048M -drive file=arm64-vm.img,if=virtio -M virt -cpu cortex-a57 \
	-kernel vmlinuz-4.19.0-13-arm64 -initrd initrd.img-4.19.0-13-arm64 \
	-append 'console=ttyAMA0,115200 root=/dev/vda net.ifnames=0 biosdevname=0' -no-reboot