It’s not often in 2021 that you find yourself building new kernels, but nevertheless, the occasion comes that you need to either enable a flag—or even worse—patch the kernel. This happened recently: on DMOJ, we recently run into a kernel issue that misreports the memory usage for processes as an “optimization.” For more information about this issue, see the excellent blog post by my friend Tudor. As a result of this, I was forced to build a patched kernel to work around this issue. Since the process was far from easy, I decided to write this blog post to help others in the future.

Building a kernel is not too difficult, actually. The real challenge comes in the form of building the kernel in a maintainable way, which basically means that we should at least build the kernel into an easily installable package. For example, on DMOJ, we manage multiple judge virtual machines, and they all need to receive the same kernel. Furthermore, we want our custom build of the kernel to be distinct from the standard kernels that the operating system offers, as we don’t want a system upgrade to undo the patch that we applied.

In this article, we will explore the process I used to build a custom kernel package on Debian for the scenario described above. This will involve both patching the kernel and subsequently changing a configuration option. Specifically, we will be applying this patch. These instructions should work with minor adaptations for other Debian-based distributions.

Getting the kernel source code

First, we want to start by downloading the latest kernel. If you simply want to rebuild the stock kernel of the current Debian release, you can download the source code complete with Debian build scripts by running:

$ apt source linux
$ cd linux-*

Backporting

However, if you want a newer kernel, you can backport it from sid. First, you need to enable the sid source code repository by adding the following snippet to /etc/apt/sources.list:

deb-src http://deb.debian.org/debian/ sid main

Note that this is distinct from allowing apt to install packages from sid.

We can then get the sid version of linux by running:

$ apt source linux/sid
$ cd linux-*

We then want to tell Debian to backport this kernel by running:

$ DEBFULLNAME=Quantum DEBEMAIL=[email protected] dch --bpo ''

The name and email are purely cosmetic, but it helps to make it look professional.

Patching the kernel

We would like to start by applying our patch, which is available as a diff file here. We can use the quilt tool to import the patch, apply it, and format it so that it applies cleanly without warning:

$ quilt import -P split-res-config.patch <(curl https://lore.kernel.org/patchwork/patch/1133454/raw/)
$ quilt push -a
$ quilt refresh

Our kernel is now patched!

Creating a new kernel flavour

This is probably the most important part. Without this step, the next time you run apt upgrade and there is a newer kernel, your patch will be undone. However, if we turned our patch into a new kernel flavour, similar to how the linux-image-cloud-amd64 package is implemented, apt will track this specific variant of the kernel and not blindly upgrade. However, you should still manually build the kernel to keep things up-to-date.

We will be calling our kernel flavour rss-amd64. To add this flavour, first, edit debian/config/amd64/none/defines in the source tree, and add rss-amd64 to flavours under the [base] section. Then, we need to add the following block to the end of the file:

[rss-amd64_image]
configs:
 amd64/config.rss

[rss-amd64_build]
signed-code: false

This tells the build scripts that our new flavour will use the configuration file amd64/config.rss (which we will create later), and we would not be signing the kernel image for secure boot.

We then need to edit debian/config/amd64/defines to tell the build scripts what description to put in the package. Simply add the following block:

[rss-amd64_description]
hardware: x86-64 precise rss
hardware-long: DMOJ judge VMs with precise RSS measurements

Finally, we need to create the debian/config/amd64/config.rss file. We just need it to contain:

CONFIG_SPLIT_RSS_COUNTING=n

Next, we need to regenerate the makefile:

$ debian/bin/gencontrol.py

Building the kernel

We simply want to build the kernel image and nothing else, certainly not the other flavours, which would take ages. To build the rss-amd64 kernel:

$ fakeroot make -f debian/rules.gen binary-arch_amd64_none_rss-amd64 -j$(nproc)

The last part, -j$(nproc), is necessary to leverage multiple processors. Without it, this build will take forever.

This will take a few minutes at least, up to an hour, depending on how fast your computer is. For reference, on my Ryzen 9 3900X, this took about 10 minutes.

Once you are done, the kernel should be available:

$ ls ../*.deb
-rw-r--r-- 1 quantum quantum 833K Aug  4 03:48 linux-headers-5.10.0-8-rss-amd64_5.10.46-4~bpo10+1_amd64.deb
-rw-r--r-- 1 quantum quantum 1.2K Aug  4 03:36 linux-headers-rss-amd64_5.10.46-4~bpo10+1_amd64.deb
-rw-r--r-- 1 quantum quantum  50M Aug  4 03:49 linux-image-5.10.0-8-rss-amd64_5.10.46-4~bpo10+1_amd64.deb
-rw-r--r-- 1 quantum quantum 925M Aug  4 03:50 linux-image-5.10.0-8-rss-amd64-dbg_5.10.46-4~bpo10+1_amd64.deb
-rw-r--r-- 1 quantum quantum 1.5K Aug  4 03:36 linux-image-rss-amd64_5.10.46-4~bpo10+1_amd64.deb
-rw-r--r-- 1 quantum quantum 1.4K Aug  4 03:36 linux-image-rss-amd64-dbg_5.10.46-4~bpo10+1_amd64.deb

Distribution

You can install this kernel simply by running:

$ cd ..
# apt install ./linux-image-rss-amd64_5.10.46-4~bpo10+1_amd64.deb ./linux-image-5.10.0-8-rss-amd64_5.10.46-4~bpo10+1_amd64.deb

If you need the headers too (e.g. for building kernel modules), you can install ./linux-headers-rss-amd64_5.10.46-4~bpo10+1_amd64.deb and ./linux-headers-5.10.0-8-rss-amd64_5.10.46-4~bpo10+1_amd64.deb as well.

However, you may have noticed that using these long file names and resolving the dependencies manually is rather annoying.

Ideally, you want to distribute this kernel in an apt repo. A very simple apt repo can be created with a few commands. We will be assuming you copied the .deb files into /srv/apt. If this is the case, we can simply run:

$ dpkg-scanpackages /srv/apt | gzip > /srv/apt/Packages.gz

We can then tell apt to use this repository by putting the following line in /etc/apt/sources.list:

deb [trusted=yes] file:/srv/apt ./

We use [trusted=yes] to tell Debian that the repository should be trusted anyways despite the fact the Release file is not PGP-signed by a trusted key.

Then, you can simply run the following commands to install the kernel:

# apt update
# apt install linux-image-rss-amd64

You can also host the repository on an HTTP file server and use the following line to enable it:

deb [trusted=yes] https://apt.example.com ./

A better apt repository including package signing can be built with reprepro. This is how I host my apt repositories. However, it is beyond the scope of this post. Some useful information may be found on the Debian Wiki.