Border Gateway Protocol (often abbreviated BGP) is a critical protocol that makes the modern Internet possible, yet remains one of its most poorly understood parts even among its long-time users. At the same time, it has played a significant role in several high-profile outages on the Internet. As someone who has been running my “own piece of the Internet”—AS200351—for half a year now, I think the time has come to write a piece explaining exactly what BGP is, what AS200351 is, and how the Internet truly functions behind the scenes. We’ll start with the basics.

To understand BGP, we must first understand why it is called the “Internet” in the first place. To simplify greatly, the Internet is called that because it’s an interconnected network of networks (more precisely, autonomous systems, or ASes) glued together by BGP. Since this sounds like a nonsensical sequence of words, let’s dive a bit deeper.

In January, I upgraded my home Internet connection to 3 Gbps symmetric, because, strangely enough, it was cheaper than the package I already had at the time (1500 Mbps down, 940 Mbps up). This was connected to the second port on my ConnectX-3, allowing my home server to achieve the full speed where 2.5 Gbps Ethernet would have failed. Unfortunately, nothing I was doing could have harnessed the full speed of this Internet connection, or anywhere near it, so I started thinking…

In February, I realized that I could run a mirroring service for open-source software to serve the community at basically no additional cost—I am already paying for this 3 Gbps Internet connection and I have some spare disk space on my SSD. So I decided to do exactly that.

Today, I am happy to announce that this mirror, mirror.quantum5.ca, has been tested for a few months and is fully ready for production. If you find the service helpful, please feel free to support me via GitHub Sponsors, Ko-fi, Liberapay, or directly with credit card or bank through Stripe (CAD), though this is of course strictly optional.

If you are interested in how it’s all set up, please read on:

Last time, I built a stratum 1 NTP server with a PPS signal from a GPS receiver, synchronizing my server’s clock to within 10 microseconds of UTC. However, NTP was designed to synchronize clocks within a few tens of milliseconds over the Internet, and I’d be lucky to achieve millisecond accuracy on a LAN. I mentioned that PTP was the alternative that could achieve accuracy in the sub-microsecond range. Well, this time I’ll be setting up PTP between my server and my PC with the hardware timestamping on the ConnectX-3s.

If you are following along at home, don’t despair if your hardware can’t do timestamping or PTP. I will also attempt to set up PTP with software timestamping later for my other devices.

Naturally, I first turned to the gpsd documentation, since that was a decent reference for setting up NTP with the PPS signal. Well, this is what it says for PTP with hardware timestamping:

Sadly, theory and practice diverge here. I have never succeeded in making hardware timestamping work. I have successfully trashed my host system clock. Tread carefully. If you make progress please pass on some clue.

That didn’t sound encouraging at all. “Oh well, I guess I am on my own here,” I thought to myself. “How bad could digging through a few man pages and random online documentation be? Worst case, there is the source code, right?”

These days, it seems like everyone is posting about turning Raspberry Pis into a stratum 1 NTP server by hooking up a cheap GPS module, most often the GT-U7 u-blox 7 clone with a PPS (pulse-per-second) signal output, whose rising edge indicates exactly the start of a second.

While this seems like a cool idea, it suffers from one flaw—while the Raspberry Pi itself almost certainly has very accurate time, getting accurate time to the rest of the network would be problematic. This is because the Ethernet adapter on Raspberry Pis before the Pi 4 was hooked up via USB, and the polling nature of USB introduces jitter, preventing the accurate signal from reaching the rest of the network. Unfortunately, I only have a Raspberry Pi 3 model B in my possession, which suffers from the problem.

Now, I could have gotten a Raspberry Pi 4, but those aren’t priced sanely at the moment and it would be just an exercise in copying. Instead, I looked at the various alternatives. The traditional way of doing this kind of thing involves hooking up a GPS receiver into a serial port, which generates an interrupt. If the PPS signal is delivered to the DCD (data carrier detect) signal (as described in RFC 2783), then the in-tree Linux driver pps_ldisc is able to do the timestamping in kernel mode for the highest possible accuracy.

I found out that my server’s X570 motherboard came with a serial port header (labelled COM). This meant that I could buy some fancy GPS receiver with a serial port and hook it up. Unfortunately, those aren’t priced sanely either, so I decided to build my own with the GT-U7 module and a driver module for RS-232 (the common serial port standard).

This was late last year. I ordered the components on AliExpress and they all arrived in January, so I finally started this project.

2022 was certainly an interesting year. While the world events were rather depressing, we are not here to talk about them. Instead, let us explore what I did this year—if just to help me remember it years down the line.

In January, I ended up messing around with my domains. I wrote about this in a previous blog post, but here’s a summary: To improve email delivery, I moved this website from quantum2.xyz to quantum5.ca. Furthermore, I saw qt.ax was open for registration, and registered it to use as my URL shortener. While switching to quantum5.ca was a relatively straightforward procedure, registering qt.ax at a rather steep price of €32/year would bring about a rather interesting sequence of events, as we shall see later.

I also talked earlier about my globally distributed backend, which in January consisted of three nodes: Montréal, Amsterdam, and Sydney. By the end of the year, this would change significantly.

In February, I implemented my own version of the French Republican Calendar for fun, which spawned a whole series of posts on the subject. At the end of the month, AMD dropped the retail price of the Ryzen 9 5950X, at which point I impulsively bought one to replace my 3900X, which would have some interesting consequences.

Last time, we discussed how we might add a real GPU to our Windows virtual machine. Today, we’ll discuss how to view this virtual machine without using a dedicated monitor or switching inputs, but instead integrating it into the Linux desktop like a normal application.

There are three steps:

1. Configuring the virtual machine.
2. Installing the Looking Glass client on the host machine.
3. Setting up Looking Glass host application on the virtual machine.

Without further ado, let’s begin.

Last time, we discussed how we might create a Windows virtual machine as part of a series on running a Windows VM with native-level graphics performance via GPU passthrough and integrating it seamlessly into your Linux desktop via Looking Glass. Today, we shall turn that normal Windows virtual machine into something far more interesting by giving it a real GPU.

As far as Windows is concerned, the GPU is real hardware and can be treated as normal, so we will not go into too much depth. Most of the work lies on the Linux side, where we must do some work to make sure the GPU is free for the VM to use, and then instruct the hypervisor to use it. Again, we will be using the standard QEMU/KVM setup, managing our virtual machines with libvirt.

Naturally, the same procedure here can be used for any other PCIe device, such as NVMe SSDs. Let’s begin!

Last time, we introduced a series on running a Windows VM with native-level graphics performance via GPU passthrough and integrating it seamlessly into your Linux desktop via Looking Glass. We start this journey by creating a basic Windows virtual machine, which will form the foundation of all future work.

For this example, I decided to use Windows 11 for fun, since I did it quite a few times with Windows 10 already. However, given that Windows 11 is basically a renamed Windows 10 with some additional hardware requirements, there is not much of a difference anyway.

On the Linux side, we will be using the standard QEMU/KVM setup, managing our virtual machines with libvirt. Let’s begin!

As you may know, I contributed quite a bit to the Looking Glass project — an ultra-low-latency viewer for virtual machines with GPU passthrough. To those who understand the use case, this is amazing technology; but to most people, these words hold little meaning. I had plenty of difficulty explaining what this is all about, so I thought I’d write about it.

What is GPU passthrough? It is essentially giving a dedicated GPU to a virtual machine, just as if you plugged it into a PCIe slot if it were a real machine. This is commonly called “VFIO” (Virtual Function I/O). Originally, this is intended for server applications, for example, giving a network card to a virtual machine. These days, however, it’s also commonly used by Linux users to run a Windows virtual machine to play games at native speeds without dual-booting — just as if you had a separate Windows computer.

Annoyingly, this requires you to plug a monitor into the Windows GPU to get a display output, requiring you to either have a new monitor, switch inputs, or buy a KVM switch. Looking Glass is meant to address this problem — by capturing the output of the Windows display and presenting it to you in a window that is integrated into your favourite Linux desktop environment. This eliminates all annoyances with monitors — simply move your mouse into the window to start using Windows, and move it out to use Linux.

In this series, I will slowly walk you through the process of creating such a virtual machine and installing Looking Glass, explaining the technical details along the way. Expect updates over the next little while:

1. Part 1: Creating a basic Windows VM
2. Part 2: Passing through PCIe Devices
3. Part 3: Setting up Looking Glass

Over a year ago, I wrote about making ARM virtual machines. Times have changed a lot since then — the release of Apple M1 has dramatically changed the perception of ARM. No longer is ARM a niche platform for low-power gadgets like phones or tablets, but a viable desktop computing platform. Similarly, in the server space, Amazon Graviton and Ampere Altra have gained traction. My old blog post presented only a quick hack to get the ARM virtual machine to boot — by copying the kernel image. This leaves a lot to be desired. Somehow, despite this, it quickly became one of the popular posts on my blog. Today, we shall rectify that flaw and present a way to boot the latest kernel installed in the virtual machine using the Unified Extensible Firmware Interface (UEFI).

Again, like before, this tutorial will use Debian as an example, but the same methodology should work for other distributions. If you are looking for a simple chroot, you should instead follow the original post.