Victor Brinkhorst on Qstars IT Blog - Cloud, Kubernetes & Software Engineering

Cheap Self-Hosted Kubernetes on Hetzner Cloud

Tue, 30 Dec 2025 09:21:00 +0200

This blog’s starter project can be found at github.com/qstarsit/hcloud-kube-hetzner

Kubernetes is a brilliant workload API, the community has built a highly extensible standard which abstracts the underlying machines and infrastructure from our application deployments. However, the cloud-native ecosystem can come with a sense of vendor lock-in. Functionality like managed load balancing, seamless node autoscaling, extensive storage options, integrated networking, and polished operational tooling are what you pay a premium for.

Running and operating Kubernetes yourself, “The Hard Way”, can be overwhelming, and time intensive. That’s why in this post, we’re taking a look at terraform-hcloud-kube-hetzner. An open-source Terraform project that simplifies deploying a production-ready K3s cluster on Hetzner Cloud, striking a balance between ease of use, reliability, and cost efficiency. All while keeping true to our European origin 🇪🇺.

Hetzner

Hetzner is a German infrastructure provider that has built a strong reputation for offering high-performance hardware at reasonable and predictable prices. With less features than hyperscalers they focus on doing few things very well. This has made them a popular hosting provider among indie hackers and startups alike.

Their Cloud offering has matured a lot in recent years and features a solid API to manage resources. It features private networking, load balancers, network attached storage and virtual machines. This makes it a solid alternative for teams looking for proper automation on a more cost-effective platform.

Hetzner operates modern data centers primarily in Europe, while also offering locations in the USA and Asia. In light of the recent focus on sovereign data centers, the fact that Hetzner is a German company adds reassurance around data sovereignty.

Kube-Hetzner

Kube-Hetzner is an optimized, easy-to-use terraform module to set up a (highly available) auto-upgraded cluster. It codifies many best practices required to run K3s reliably on Hetzner Cloud.

The project handles much of the heavy lifting involved in bootstrapping and maintaining a cluster: provisioning compute resources, configuring private networking, setting up load balancers, managing node pools, and wiring it all together. Upgrades are automated and rolling by default, reducing manual intervention and downtime during maintenance.

It integrates Hetzner’s native tooling directly into your cluster, including the hcloud CSI driver and the hcloud Cloud Controller Manager (CCM). This enables Kubernetes to automatically provision and manage resources like load balancers and block storage via Kubernetes.

MicroOS

In the world of containers, less is more. The same applies for the operating system that functions as the base for our workloads. Traditional server distributions like Ubuntu, Debian or RedHat Enterprise Linux include a wide range of packages and services for general-purpose use, many of which are unnecessary for container workloads. This has led to the development of container-centric operating systems; Talos, CoreOS, Bottlerocket and flatcar just to name a few.

Kube-Hetzner utilises openSUSE’s MicroOS for all of its nodes. MicroOS is a stable operating system with a small footprint specialized in running container workloads. It sports rolling releases, atomic & transactional updates and secure updates by default. The OS is designed to be predictable, resilient and to behave the same on every boot.

MicroOS is immutable and cannot be modified during runtime. Updates are applied transactionally using Btrfs snapshots, allowing the system to fully succeed or roll back to a clean state. In the event of a faulty update a previous snapshot is automatically booted with no manual interaction required.

A read-only root filesystem also reduces our attack surface. After a container is compromised, modifying host binaries or operating system configuration is not possible. Even if an attacker gains host-level access, any changes are discarded on reboot.

Our application workloads are managed independently from the host operating system and are not installed on the root filesystem. This prevents configuration drift and ensures all of our nodes remain interchangeable and can soundly be rolled out, replaced or recovered repeatedly.

K3s

The heart of our cluster consists of K3s, Rancher’s lightweight Kubernetes distribution. Designed specifically for compute-constrained environments like edge or IoT devices. K3s packages components like the Kubernetes API server, kube proxy and kubelet into a single binary to reduce dependencies and steps required to bootstrap a cluster.

It’s a fully conformant distribution that simplifies our operations. Despite its minimal design, K3s is production ready and well suited for small to medium-sized clusters. It compacts and abstracts a lot of the overhead that comes with running a standard Kubernetes installation.

Furthermore, K3s and Kube-Hetzner come with sensible defaults. You are free and able to change advanced configurations in your cluster such as the IP ranges used for the pod and service network or change the internal cluster hostname. More on these settings later as we’ll configure them with Terraform.

K3s server configuration

Planning our cluster

For this blog we’ll deploy a standard highly-available cluster. Three control plane nodes and three worker nodes spread out across Falkenstein, Nuremberg and Helsinki.

For cost efficiency, we’ll enable the Rancher local-path provisioner to make use of the disks attached to our VPS nodes. This lets us choose between regional block storage or disks attached to our nodes using local-path, depending on the workload.

I’ll opt out of using Longhorn as the local-path provisioner combined with the block storage is sufficient for my workloads. If you require storage that is replicated across regions and independent of pod scheduling decisions, you can enable Longhorn. Do beware of performance bottlenecks when using Longhorn and benchmark it for your use case.

💡 Alternatively, you could opt for a dual-region failover setup across the two German sites. While the Helsinki region generally has good server availability and acceptable latency for production workloads, its latency is still higher than the site-to-site connection between Falkenstein and Nuremberg.

Select your desired machine type for the control plane and worker nodes. I’ll opt for CX23 control planes (2 virtual cores / 4GB ram / 40GB disk @ €5,49/mo) and CX33 (4 virtual cores / 8GB ram / 80GB disk @ €8,49/mo) for our workers. These instances are part of Hetzner’s shared cloud offering, meaning resources are shared with other customers on the same physical host. In my experience, I haven’t observed significant CPU steal time and do not believe that Hetzner aggressively oversubscribes their servers. That said, you should evaluate this choice carefully based on your specific workload and business requirements.

💡 At the time of this update, CX23 and CX33 are showing as temporarily sold out in most EU locations due to high demand on Hetzner’s shared vCPU lines. This has historically been intermittent — check the Hetzner Cloud console or try a different location (Falkenstein, Nuremberg, Helsinki) before reaching for a pricier dedicated-vCPU type.

That puts us up to €41,94 a month on our six machines so far. Now we need to decide on a network configuration for our cluster.

For this post, I’ll keep it simple. We’ll opt to give each node a public IP address and whitelist our IP address to connect on port 22 (SSH). The API Server will be closed for external traffic as we’ll put a load balancer infront of our control planes.

Additionally, our traefik ingress class gets its own load balancer. This gives us a highly available, seperate entrypoint for cluster management and application traffic. These two load balancers (lb11) cost €7,49/mo each, adding €14,98/mo to our total.

A public IPv4 address now costs €0,50/mo which adds an additional €3,00 making our total €59,92/mo pre-VAT and €72,50/mo with VAT.

To me, this remains a reasonable cost for our cluster, though the margin has narrowed since Hetzner’s 2026 price adjustments. Barring promotional discounts and free tiers, a typical managed cluster starts at $70-$75/mo on GCP GKE or AWS EKS — comparing against our pre-VAT total (VAT-registered businesses reclaim it), we’re still meaningfully cheaper, but not by the wide margin we used to be. That is just the cost of running your cluster, not the workloads you are going to schedule on it.

However, there are some options for those looking to push down the costs further:

1. Use a single load balancer or MetalLB

Instead of using separate load balancers for the API server and ingress traffic, you could consolidate to a single load balancer that handles both. This saves €7,49/mo pre-VAT, bringing the total down to €63,44/mo with VAT.

Alternatively, skip Hetzner’s load balancers entirely and use MetalLB as a software load balancer running inside your cluster.

2. Remove public IP addresses and use private networking

Each public IPv4 address now costs €0,50/mo, totaling €3,00/mo for six nodes — a much smaller saving than it used to be, now that Hetzner has cut IPv4 pricing. By removing public IPs and connecting to your cluster through a bastion host, NAT gateway, or VPN solution, you still improve your security posture, even if the direct cost savings are now marginal.

Options include:

Bastion host: A single small node with a public IP that you SSH through to access cluster nodes
WireGuard VPN: Set up a WireGuard server on one node or a separate instance to provide secure private network access
Tailscale/Nebula: Use a mesh VPN solution for seamless private connectivity

This approach removes €3,00/mo in IP costs, though the main benefit today is the reduced attack surface rather than the savings.

Rolling out our cluster

It’s time to actually deploy the cluster. Head over and clone our starter project to get started. Take note of the Important configurations section in the Readme file and change the values to your liking.

Now we assume that you have Terraform, Kubectl and the hcloud CLI installed. If you do not have these, install them now.

In the project directory, start by installing the terraform dependencies by running terraform init -upgrade.

Next we plan our infrastructure changes with terraform plan. Take a moment to look over the plan, it shows you exactly what terraform is going to create for you in Hetzner. If you’re satisfied with the planned changes apply them with terraform apply -auto-approve. The applying process can take up to 10 minutes.

💡 Getting stuck waiting for MicroOS to become active in the local-exec step during the apply? Make sure your machine has connectivity to the nodes and you’ve whitelisted your IP address for SSH connections.

After a successful apply we can get our kubeconfig with terraform output -raw kubeconfig > config. Move the config file to the default .kube directory ~/.kube/config for Linux & MacOS or %USERPROFILE%\.kube\config for Windows.

Now we can test connectivity with our cluster by running kubectl get nodes.

Success! My 6 node cluster has successfully rolled out with my nodes being spread over the three Hetzner zones. Checking my ~/.kube/config file I can see I am connecting to https://142.132.242.235:6443.

This should be the IP address of the controlplane load balancer, let’s verify it in the Hetzner UI.

Perfect! All resources have been created successfully. Our cluster is now up and running with:

6 nodes spread across 3 availability zones
2 load balancers for API server and ingress traffic
Private, encrypted networking between all nodes
Automated K3s installation and configuration
Ready to deploy workloads

From here, you can start deploying your applications using kubectl apply or install additional tooling like monitoring, logging, or CI/CD integrations. The cluster is production-ready and will automatically handle upgrades and maintenance through the built-in controllers we discussed earlier.

Backups

Kube-Hetzner makes use of K3s’ built-in etcd snapshot capability. This allows us to backup our cluster state to an external S3 compatible storage provider. K3s automatically creates etcd snapshots on the nodes themselves, relying solely on local snapshots leaves us vulnerable to a catastrophic node failure.

Node backups are facilitated through Hetzner’s built-in backupping mechanism.

Upgrades

KH uses Rancher’s System Upgrade Controller (SUC) to manage Kubernetes and node-level upgrades in a safe, fully automated way. SUC enables upgrades to be defined declaratively and executed natively within Kubernetes.

The controller continuously watches for Plan resources. When an upgrade is required, it selects the appropriate nodes and coordinates the rollout according to the defined strategy. This includes handling upgrade order, limiting concurrency, and applying cordon and drain rules. Each Plan clearly defines what is upgraded, how the upgrade is performed, and which nodes are affected.

Complementing the SUC, KH also deploys kured the Kubernetes Reboot Daemon. Kured handles node restarts when the OS signals that a reboot is required. Kured monitors the reboot signal and coordinates reboots across nodes managing concurrency and cordoning the nodes. You can tell kured to only reboot your nodes within set timeframes and on specified days only.

Recap EuroPython 2025

Mon, 08 Sep 2025 10:00:00 +0200

EuroPython is a full week dedicated to all things Python. This year the conference ran from 14–20 July in historic Prague.

My first EuroPython

This was my very first EuroPython. Right after picking up the badge, you’re welcomed by a table full of stickers to customize it — interests, working groups, and even a “first EuroPython” tag. It was fun to see such a mix of long-time regulars and newcomers like myself.

Talks

When I first opened the schedule, I was a little overwhelmed. With more than 180 speakers and around 100 talks, it wasn’t easy to choose where to start. One thing was clear: boredom wasn’t a part of the programme.

The conference offered variety on every level — speakers, topics, and presentation styles. From catching Ukrainian snacks while discussing football analytics (thanks, Ruslan), to learning about Polish strawberry pasta recipe during a deep dive into concurrency controls. Each speaker, whether a first-timer or a veteran, brought a piece of themselves to the stage, making every talk all the more engaging.

Beyond the sheer number of talks, the range of topics was amazing. One moment you’re diving into the internals of FastAPI and Django, and the next you’re building your own ASGI-compliant protocol. The programme spanned everything from DevOps, Kubernetes and geospatial engineering to GPU-accelerated computing, microcontrollers, and of course, AI. Whatever your area of expertise, if you work with Python, chances are you’ll find someone tackling similar challenges at EuroPython.

Below is a selection of talks I’d like to highlight.

Myths and fairy tales around Python performance - Antonio Cuni

Antonio Cuni, principal software engineer at Anaconda and author of SPylang. SPy is subset of Python designed to be statically compiled. The project aims to balance the ease-of-use and productivity of a dynamically typed language, while stripping the language of some features to make it faster.

In his talk, Antonio covers some of the most common misconceptions around Python performance. He began with a comparison of the different layers of memory, from the very fast L1 cache to the much slower main RAM, to illustrate why optimizing for CPU efficiency usually matters more than chasing small micro-optimizations. He then showed how Python features translate into CPU instructions, explaining why compile-time inlining is not possible given the language’s semantics and why a simple attribute call like foo.x() is more expensive than it might seem.

Intuition vs. Reality: Surprising Truths in Python Performance - Adrien Cacciaguerra & Arthur Pastel

Adrien and Arthurs are both co-founders of CodSpeed, a Continuous Performance Analysis solution. Their talk focusses on performance analysis and shows us that our intuitions about code performance can actually make our programs perform slower.

For example, in Python both lists and generators let you iterate over sequences, but they work very differently. A list is eager: when you run [x for x in range(10_000_000)], all ten million elements are created and stored in memory immediately. In contrast, a generator is lazy: it produces values one at a time, yielding them on demand while only keeping a small amount of state in memory. This makes generators far more memory-efficient than lists.

However, memory efficiency doesn’t always translate to raw speed. Each time a generator yields, the interpreter must resume its frame and execute Python bytecode, which adds per-item overhead compared with iterating over a list stored contiguously in memory. That said, generators avoid allocating and populating a full intermediate list, so in single-pass operations (for example sum(x for x in range(...))), they can actually be faster overall.

In addition to the insights, they showcased their LLM-based performance analysis platform P99 Chat. In P99 you can upload a snippet of code and have the AI write a summary of how well it’s performing or where it can be tweaked.

Breaking HTTP Boundaries: ASGI to build a fleet management system in Python - Kirill Druzhinin

The ASGI (Asynchronous Server Gateway Interface) protocol is the successor of WSGI. ASGI provides a standard interface between async-capable Python network protocol servers, frameworks and applications. It’s most commonly known for it’s use with web applications, but you can do way more with the protocol.

Kirill’s demo went beyond the usual HTTP boundaries, showing how ASGI can be used for custom protocols and real-time communication. Unlike HTTP, which is typically request-response and half-duplex, ASGI supports full-duplex communication—meaning data can flow in both directions simultaneously. This opens up possibilities for multiplexed protocols, streaming, and more advanced use cases (websockets or custom IoT protocols).

He explained how his protocol server processes incoming frames, checking if they arrive sequentially and handling them according to the protocol’s rules. The server acts as a bridge between the network layer and the Python application, ensuring that each frame is correctly routed and that the client and app stay in sync. Features like more_body=True in ASGI allow for streaming data, making it possible to handle large payloads or continuous updates efficiently.

The community

What stood out most was the community (how clichë). Python’s culture is just as approachable as the language itself: people are eager to help at every level, whether you’re learning Django for the first time or digging into the CPython JIT.

This was refreshing compared to other programming communities where beginners can get dismissed as or feel like they’re “not good enough”. I started teaching myself programming when I was about 12, there were plenty of moments when I felt like giving up for a plethora of reasons. It means the world to me to see how patient and supportive people at EuroPython are.

It felt like an environment where learning and socializing come naturally — where your intelligence or worth isn’t measured by your seniority in a particular field.

I say all of this to give Python and it’s community the flowers it deserves for fostering an amazing community of positive and like-minded people.

The origin story

At the closing ceremony, we were treated to a twenty–minute preview of the newly released Python documentary: An origin story. The film traces Python back to its beginnings, with Guido van Rossum himself narrating how the language emerged during his time at CWI — the Dutch national research institute for mathematics and computer science.

During the doc, notable figures like Brett Cannon and Benjamin Peterson share their perspectives on contributing to the ecosystem and how much they gained from the commuity. What struck me was how the documentary highlights not just the technical milestones, but also the people who shaped Python’s growth.

It’s a fascinating look at how a personal project evolved into one of the most ubiquitous programming languages in the world, growing in both maturity and popularity while fostering it’s community-driven spirit.

Closing words

Overall, EuroPython 2025 was a fantastic experience. The event was well organized, the catering was excellent, and the staff were very friendly. Behind the scenes, the conference depends on a dedicated team of volunteers who make everything run seamlessly. A heartfelt thank-you to all of them for giving up their own time to ensure talks and operations went off without a hitch.

Local Kubernetes on Mac: a Multi-Node Cluster with UTM

Fri, 11 Jul 2025 09:38:00 +0200

Today we’re going to configure a Kubernetes cluster using virtual machines on our MacOS host. We are going to use UTM which utilises QEMU, a type-2 hypervisor to actually virtualise the new machines. I will be using a 2024 Macbook Pro with the Apple Silicon M4 chip, this guide can also be followed on older M-chips or Intel CPU’s.

Virtual Machines vs Dockerized Kubernetes

There is an abundance of easy to use tools out there to quickly spin up a local Kubernetes cluster. You may have heard of Minikube, KinD (Kubernetes-in-Docker) or K3S. Running Kubernetes inside actual VMs simulates a more realistic production environment. Each VM has their own kernel and networking stack, this doesn’t just offer a higher level of protection to your host machine it also allows us tighter controls over the OS & Kernel.

We’re going to be installing the Kubernetes (v1.36*) cluster using kubeadm, meaning that all of our control plane components are going to be running as Pods on the cluster (in the kube-system namespace, more about that later).

*At the time of writing (August 2026), v1.36 is the latest minor release; v1.32, used when this guide was first written, has since reached end of life. Other minor versions can be installed following this guide, you just need to swap out the version numbers where specified. Learn more about Kubernetes versioning.

💡 This guide’s screenshots were captured against Kubernetes v1.32, Alpine 3.21 and Ubuntu Server 24.04. The commands below have since been updated to currently supported versions, so the exact version strings you see on screen may differ slightly from the screenshots — the installation flow itself hasn’t changed.

Prerequisites

Don’t know your CPU architecture? Run uname -m in a terminal of your choice.

💡 Got an Apple Silicon M-chip (arm64)? Use aarch64 where there is no specific arm64 download available.

A MacOS machine (Intel or Apple Silicon) running macOS 13 (Ventura) or later — required by current UTM releases
UTM
Ubuntu Server (24.04 LTS or newer — 26.04 LTS is the current release) & Alpine Linux Standard (3.24+) ISO files matching your cpu architecture

Installation

This section will cover the installation of three virtual machine types. The Router will provide internet & inter-node connectivity to our Kubernetes cluster, using a router like this will allow us to connect to any internet device without the inter-node network being disrupted (i.e by changing IP addresses or firewall rules).

The control plane and worker node(s) are standard Kubernetes components, in this guide we’ll install one worker and one dedicated control plane node. Repeat the installation steps to add more control plane or worker nodes to your cluster for high-avalability.

❗The amount of RAM (memory) is expressed in Mebibytes. The amount of processor cores refer to virtual CPU cores assigned to the VM. On Apple Silicon, these are distributed across efficiency and performance cores, while on Intel Macs, they may correspond to physical or logical hyperthreaded cores.

Router

Resource	Value
Operating system	Alpine Standard 3.24+
Memory	512 Mebibyte
Processor cores	1
Disk size	10 Gigabytes

Let’s get this party started by setting up our Router VM. Press the + icon in UTM and select Virtualize. On the next screen, select Linux

On the Linux virtualisation page we are going to leave Apple Virtualization & Boot from kernal image off. Press Browse... and select the Alpine ISO we have just downloaded.

Fill in the memory, CPU core requirements & storage as per the values above. Leave OpenGL acceleration turned off. We are going to skip the Shared Directory page.

Now you should be greeted with the Summary page, here you can review what we’ve just configured before spinning up the vm. Give it a name of your liking and press Save.

Before we can continue installing Alpine on the machine we have to add a Serial (terminal) output to the vm. Right click on the machine in the lefthand overview and press Edit.

Head over to the Devices tab, we are going to remove the primary Display that’s configured for the machine and instead add a Serial.

We’re ready to install Alpine Linux! Hit Save & press the play button to spin up our virtual machine. After waiting for some seconds you’ll be greeted with a login screen, login with the root username and no password.

We can begin the installation process by running the setup-alpine command in the terminal. We’ll be prompted by the setup wizard to configure a hostname, use a name that you’ll be able to identify the machine with, I choose router. For the Interface (networking) configuration, you can simply press enter. We want to configure the default interface to be assigned an IP address from our internet modem using DHCP.

Next, enter a memorable password for the root user. This is the most privileged user on the machine and it can perform any action, make it secure! After filling in your password you’ll be asked what timezone you’re from. From this part onwards, you can safely use the default values for sections Proxy, Network Time Protocol, APK Mirror and User. This will leave us to configure the system disk.

Wait for the installer to finish and then exit out of the vm. We are going to go back into the settings for the vm by right clicking on it in our overview and pressing Edit (if the vm isn’t stopped yet, press Stop first to make sure it’s turned off!). Now, under the Drives section, select the USB Drive and delete it, this will make sure the installation image is no longer mounted when starting the machine.

Make sure to save your changes! You’ve now installed Alpine Linux on a virtual machine. Turn on the vm and login with your previously configured password for the root user. We’re going to update our virtual machine, but before we can do that we have to make sure we have the right repositories set.

In the terminal, run the following command: cat /etc/apk/repositories It’s important to set the official Alpine main and community repositories as seen above. If you do not see these two repositories, you can add them with the command below.

`1`	`echo -e 'http://dl-cdn.alpinelinux.org/alpine/v3.24/main\nhttp://dl-cdn.alpinelinux.org/alpine/v3.24/community' > /etc/apk/repositories`

Now, we’re ready to update our system. Run the apk update && apk upgrade -y command and wait for it to complete. We’re going to have to completely turn off the vm one last time to edit the network configurations. Our goal is to have our own isolated inter-node Kubernetes network that also has a gateway to the internet.

To achieve this, we’ll have to configure two network devices for our Router. A Kubernetes Host-only network device and one that’s set to the Bridge mode. This will allow our Kubernetes nodes to direct their internet requests via the router. Let’s dig into the machine configurations first.

Under Devices you’ll see a Network device. Select it and simply change the Network Mode from Shared Network to Bridged (Advanced). Select en0 as the Bridged Interface.

Add a new Network device and set it’s Network Mode to Host-Only. Save your configurations and spin up the machine. After you’ve authenticated in the vm, run the ip a command to list the network devices. We can ignore the lo device, it’s irrelevant to us. The eth0 network device already has ip address 192.168.0.171 assigned, this is an ip address in my local home network. It’s time for us to install some required networking related packages to turn this machine into a router.

Run the following command to start installing and configuring the necessary packages, and wait for the machine to reboot.

apk add iptables && \
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE && \
/etc/init.d/iptables save && \
rc-update add iptables && \
echo 'net.ipv4.ip_forward = 1' > /etc/sysctl.d/router.conf && \
reboot

We’re almost done with the router now. All we need to do is configure the host-only device and give it a static IP in our network. I’m going to go with the 192.168.100.0/24 subnet, where 192.168.100.1 will be our router, and we will increment per Kubernetes node.

First, let’s check in on our current network interfaces configuration. Run the cat /etc/network/interfaces command. As you can see, two of our network devices have default configurations to request IP addresses. The loopback interface lets the system communicate with itself using IP address 127.0.0.1, it allows the system to talk to itself without needing a physical network device.

In Bridged mode, the eth0 interface acts like a physical device directly connected to your home network. This means it gets an IP address from your home router’s DHCP server, just like any other device on your network (e.g., your laptop or phone).

As you can see, there is no entry for the eth1 interface yet. This is the also the reason why we didn’t see an IP address assigned to the eth1 interface previously. Let’s set it up!

Run the following command in your terminal to set default configurations for eth1

`1`	`printf "auto eth1\niface eth1 inet static\n\taddress 192.168.100.1\n\tnetmask 255.255.255.0\n" >> /etc/network/interfaces`

Verify that our eth1 entry is now added to /etc/network/interfaces Apply the configuration by running rc-service networking restart. Voilà! You now have an Alpine Linux router virtual machine. Everything has been set up as it should, to verify our eth1 interface now has a static ip address of 192.168.100.1/24, run the ip a command one last time. To quickly recap: We’ve just installed Alpine Linux on a virtual machine, configured the box to allow the forwarding of network packets from our Host-Only network to our Bridged network and ultimately the internet.

ℹ️ Make sure you always have the router running alongside your Kubernetes control plane nodes. The machines that make up your Kubernetes cluster live inside the Host-Only network on your Mac, allowing them to talk to eachother but not your home router for internet access. If you notice an issue with internet connectivity, restarting the router is a great troubleshooting starting point.

It’s time for us to move on to the first Kubernetes node.

🛜 Switching Wi-Fi networks (e.g Office to Home)? Either run rc-service networking restart or reboot the router to make sure it’s assigned a new IP and connected to the new router.

Control plane

Resource	Value
Operating system	Ubuntu Server 24.04+
Memory	2048 Mebibyte
Processor cores	2
Disk size	30 Gigabytes

In Kubernetes, a node refers to a machine running a Kubernetes distribution. You have control plane and worker nodes, we’re going to start off with the control plane as it’s the brains of the operation and requires the most installation efforts of the two.

Start by repeating the three first steps to setting up a new virtual machine. Give it a name, select the operating system installation image (be sure to select Ubuntu!) and set the CPU, memory and disk specifications as outlined above.

Toggle the Open VM Settings checkbox and hit Save. In the machine settings, swap out the Display for a Serial again, and set the Network device to Host-Only mode. Save your settings and spin up the control plane.

At first you will be greeted with some fairly standard installation options, continue the installation in rich mode before selecting your language and geographical location.

When asked what base you’d like to use, keep the default Ubuntu Server base.

The next screen is important, it’s the network configuration for our control plane. Select the network device, it should only be one as we configured the default device to sit in the Host-Only network. In the new dropdown box select Edit IPv4.

Set the IPv4 Method to Manual and copy the values you see below. This will assign the control plane IP address 192.168.100.2 and use our router 192.168.100.1 as it’s gateway address.

Before hitting Save, in the Nameservers tab, fill in 1.1.1.1. Now you can save the network configuration! For the next three steps up until Profile configuration you can safely use the default values.

In the profile configuration, set the machine’s name to controlplane. Fill in your own username for the user account along with a secure password.

The next pages can safely be skipped using the default configurations. Wait for the system installer to complete before fully turning off the machine.

Once the vm is fully turned off, edit it and remove the USB Drive from it’s attached Devices again. Afterwards turn on the control plane again and login using your previously chosen username and password.

We’re going to need access to the entire system while installing Kubeadm. To get privileged access to the machine, switch to the root user by running sudo su - and typing in your password.

Let’s test our internet connectivity by saying hello to google.com using a ping. Beautiful! Our control plane machine was able to find google.com on the internet when the machine itself doesn’t have an internet connection. The control plane uses the router to access the internet. To test this, fully shutdown the router and run the same ping command again. As you can see, the machine cannot even resolve an ip address for google.com, this is because the nameserver we have configured is Cloudflare’s 1.1.1.1, in order for us to resolve a new domain name we need an internet connection in order to do so.

With the network sorted we can pivot our attention to setting up the node for Kubernetes. Run the following command to set up the required system settings & kernel modules.

sudo swapoff -a && \
sudo sed -i '/\/swap.img/ s/^/#/' /etc/fstab && \
sudo systemctl mask swap.img.swap && \
sudo sysctl -w net.ipv4.ip_forward=1 && \
echo 'net.ipv4.ip_forward=1' | sudo tee -a /etc/sysctl.conf && \
sudo sysctl --system && \
sudo modprobe overlay && \
sudo modprobe br_netfilter

We’ve enabled IPv4 Forwarding, the overlay & br_netfilter kernel modules and disabled memory swapping.

That’s not all though, we need to install a Container Runtime Interface before we can bootstrap a Kubernetes cluster on the machine. We’ll use containerd, you can install containerd from source or using a package repository. We’re going to use the latter.

The containerd team does not publish their updates to package repositories directly, this is done by the Docker team. Docker uses containerd internally and they also publish the apt packages. This means that we’re going to have to add the Docker apt repository as a source for our machine to find packages from.

sudo apt-get update && \
sudo apt-get install -y apt-transport-https ca-certificates curl gpg && \
sudo install -m 0755 -d /etc/apt/keyrings && \
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc && \
sudo chmod a+r /etc/apt/keyrings/docker.asc && \
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null && \
sudo apt-get update

That was a long command! It pulled Docker’s GPG key that’ll be used to check if packages we download really come from Docker, afterwards we update the available package list with an apt-get update.

Now the containerd.io package should be available for us to be installed, run apt install containerd.io to install it!

Verify that the installation went successful by running.systemctl status containerd, you should see the service is in an active (running) state. Do you see our previously installed kernel driver being executed? ExecStartPre=/sbin/modprobe

Run the following commands to generate and tweak the default containerd configuration.

1
2
3

containerd config default > /etc/containerd/config.toml && \
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml && \
sudo systemctl restart containerd

With our CRI installed we’ve finally completed the necessary prerequisites to transform this vm into a Kubernetes control plane, so let’s do that! Just like with the Docker repository previously, we have to download the official Kubernetes apt repository GPG key to verify we are installing the legitimate packages. Run the following command to do so;

1
2
3

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.36/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/Kubernetes-apt-keyring.gpg && \
echo 'deb [signed-by=/etc/apt/keyrings/Kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.36/deb/ /' | sudo tee /etc/apt/sources.list.d/Kubernetes.list && \
sudo apt-get update

We’ve now updated apt to also look for packages inside the Kubernetes repository. To check what packages and their versions are available to us, run apt-cache madison <package>. Where <package> can either be kubeadm, kubelet or kubectl (we’re going to be installing all three!). These packages follow the Semantic Versioning standard. We have three packages with major release 1, minor release 36 with a handful of varying patch versions. We aren’t too worried about the specific patches, let’s just go with the latest one available (1.36.2 at the time of writing).

Run the sudo apt install kubelet kubeadm kubectl -y command to install the required Kubernetes binaries. Afterwards, pin their versions to make sure you don’t accidentally upgrade any of them using sudo apt-mark hold kubelet kubeadm kubectl.

(Optional) Let’s quickly verify the versions of the installed components directly. Hm… the kubectl command shows us two versions and a connection error. Thankfully we see that the Client Version is set to our target version. The client version refers to the version of kubectl we have installed, which as per our apt command earlier is 1.36.2. Kustomize is a Kubernetes configuration manager which allows you to update manifest files without a templating language like Helm’s chart templates. This isn’t relevant to our installation however, so we can safely ignore the Kustomize version.

That leaves us with a connection refused. Kubectl automatically tries to connect to the kube-apiserver on localhost:8080 if no alternative destination is configured. In our case this fails because we haven’t got the control plane components running yet.

It’s finally time for us to bootstrap a Kubernetes cluster using kubeadm. Let me introduce you to two very important initialization flags. The --service-cidr flag lets you set a subnet that Kubernetes will use for all of its services. The --pod-network-cidr lets you specify a subnet that will be used to assign IP addresses to pods.

It’s important that both of these subnets are not being used on your network! Thankfully, our Kubernetes cluster lives inside our Mac’s Host-Only network and does not automatically have access to a wider network, limiting the chance for a collision, however it’s still a good idea to decide on a dedicated subnet. I am going to use the default service cidr, and I’m going to specify 192.168.0.0/16 as the pod network cidr.

🛜 Be careful and don’t randomly assign a subnet to your cluster! It’s best to use a private subnet as your pod cidr, as per RFC 1918 192.168.0.0/16 is a private subnet. 10.96.0.0/12 (default service cidr) is technically not private, but practically it is so we’re fine with keeping the default.

That leaves us to run the kubeadm init --pod-network-cidr=192.168.0.0/16 command to get our cluster started.

⚠️ Having issues getting kubeadm init to work? Check if cri isn’t a disabled plugin in containerd, run cat /etc/containerd/config.toml | grep disabled_plugins. If you see disabled_plugins = ["cri"], scroll down to the Troubleshooting > Containerd CRI Disabled Plugin section of this post before retrying the init command and continuing.

If all is well, your terminal output should look something like this; That’s a lot of information, let’s laser in on what we actually need.

Our main area of concern here is the long kubeadm join command at the very bottom, save this in a note. As you can see, it points to our control plane machine 192.168.100.2 at port 6443. We haven’t explicitly opened a port, or a ran an application that binds to that port ourselves… Run the export KUBECONFIG=/etc/kubernetes/admin.conf command followed by kubectl get pods -A. We just ran the command to list all pods in every namespace. As we’ve just bootstrapped this cluster using kubeadm, our control plane components are ran as pods inside the kube-system namespace. As you can see, the coredns pods are in a Pending state. Let’s have a look at how our control plane node is doing. Our control plane node has a status of Not Ready, this is because CoreDNS is not running. Our CoreDNS pods will stay in this pending state until we have installed a Container Networking Interface. There’s a couple of options to choose from such as; Flannel, Calico and Cilium. Pick whichever one you prefer, I’m going to pick Cilium.

ℹ️ The Flannel CNI is only a layer 3 networking solution for Kubernetes. Many Kubernetes features such as network policies are not supported and do not take effect when using Flannel.

Though we can install Cilium using Kubernetes manifest files, they have made it easy for us to get everything set-up with their own installer. Let’s install the Cilium CLI.

💡 Preparing for the CKS exam? Cilium is the CNI used on the exam, by installing Cilium in your local cluster you can fully utilise it’s features such as the eBPF based network stack, Cilium Network Policies (layer 3, 4 & 7) and Mutual Authentication (mTLS). Keep your eyes on the exam curriculum to stay up to date.

CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt)
CLI_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}

We now have access to the command cilium. We can install Cilium onto our cluster using the cilium install command. I am going to install it with the SPIRE server enabled which is required for mTLS.

1
2
3

cilium install \
--set authentication.mutual.spire.enabled=true \
--set authentication.mutual.spire.install.enabled=true

So far so good, let’s check on our Cilium installation with the cilium status command. Everything works! We can verify this one more time by running kubectl get pods -A again. Look at that! Cilium is installed and is running it’s pods, our CoreDNS pods are now also in a Running state. Let’s check on our node again to see if it’s changed it’s status using kubectl get nodes. Perfect! Our cluster is up and running! Remember that kubectl version command that was giving us the connection error from before? No more connection issues, we now have a working kube-apiserver to handle the request. However, in it’s current state our cluster isn’t going to run any of our applications just yet. Our pod is stuck in a Pending state. This is because our control plane machine has a taint on it that prevents us to accidentally schedule a workload on the control plane. It’s a best practice to keep your control plane isolated from any of your other applications, both for cluster security and to ensure your control plane has enough resources to function. Control planes come with the node-role.kubernetes.io/control-plane taint which has a NoSchedule effect. If we really wanted to, we can remove this taint and all of our applications are able to be scheduled freely on the control plane. You can also choose to control what applications can run on the control plane by giving it a toleration.

We’re going to follow the best practice and leave the control plane as is, it’s time to set up a worker machine.

Worker(s)

Resource	Value
Operating system	Ubuntu Server 24.04+
Memory	1024 - 4096 Mebibyte
Processor cores	1 - 4
Disk size	30+ Gigabytes

Our worker nodes are going to be the ones actually running our applications. Usually your worker nodes are bigger and more powerful than your control plane node, as in my case I am not going to be hosting any large applications on my cluster I have equipped my worker node with the minimal needed system resources. If you need a more powerful test environment, consider allocating more system resources to your workers.

In UTM, create a new virtual machine with the same Ubuntu base image as the control plane node. Remove the Display from the Devices section and don’t forget to change the Network to Host-Only.

Spin up the machine and follow the same installation steps as the control plane. Don’t forget to change the assigned IP Address in the network tab to 192.168.100.3. Set the same Cloudflare 1.1.1.1 nameserver.

You can safely use the defaults for all the installation steps, just fill in the device Profile section with your username and password again.

After completing the installation wizard, fully turn off the VM and remove the USB Drive from it’s Devices, remember? Start up the machine again and switch to the root user sudo su -. Update the system packages apt update && apt upgrade -y. To get our worker to join the cluster, we need to perform the same tweaks and installations as the control plane node. Copy and paste the following command to enable kernel modules, turn off swap, allow for ip forwarding and install the Kubernetes distribution binaries (I’ve chained all of our previous commands into one!).

sudo swapoff -a && \
sudo sed -i '/ swap / s/^/#/' /etc/fstab && \
sudo sysctl -w net.ipv4.ip_forward=1 && \
echo 'net.ipv4.ip_forward=1' | sudo tee -a /etc/sysctl.conf && \
sudo sysctl --system && \
sudo modprobe overlay && \
sudo modprobe br_netfilter \
sudo apt-get update && \
sudo apt-get install -y apt-transport-https ca-certificates curl gpg && \
sudo install -m 0755 -d /etc/apt/keyrings && \
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc && \
sudo chmod a+r /etc/apt/keyrings/docker.asc && \
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null && \
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.36/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/Kubernetes-apt-keyring.gpg && \
echo 'deb [signed-by=/etc/apt/keyrings/Kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.36/deb/ /' | sudo tee /etc/apt/sources.list.d/Kubernetes.list && \
sudo apt-get update && \
sudo apt install containerd.io kubeadm kubelet kubectl -y && \
containerd config default > /etc/containerd/config.toml && \
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml && \
sudo systemctl restart containerd kubelet

Remember that kubeadm join command you saved from before? We’ll run it on our worker node to join the cluster (you can omit the --v=5 flag). Success! Our worker node has joined the cluster according to kubeadm, on our control plane we can run kubectl get nodes to have a look at the worker status. We can see our worker node has joined the cluster and is ready! By default, when joining a cluster, the kubeadm join command does not assign a role label to nodes. If you would like to explicitly label your node with the worker role run kubectl label node worker node-role.kubernetes.io/worker=. Functionally this doesn’t make a difference for us, but it’s nice to have. You can substitute the role of your node with anything you like. This allows you to dedicate nodes per role in your landscape. Instead of categorising your nodes as control plane & worker nodes, you can separate workers based on their workloads (i.e database, backend, etc.) Running kubectl get pods -A -o custom-columns="NAME:.metadata.name,NODE:.spec.nodeName,STATUS:.status.phase will show us all of the pods running on the system and on which node they have been scheduled. We can see that the necessary Cilium and kube-proxy pods are running on the new worker node. And look at that, our nginx image is now out of it’s limbo state and is running on the worker node.

🎉 Congratulations! You now have a fully functioning, host-isolated Kubernetes cluster based on virtual machines. Adding more workers or control planes for high availability is as easy as just repeating the steps we’ve taken previously. Just make sure you add the --control-plane flag on the kubeadm join command for additional control planes.

Challenge: Connect Your Mac to the Isolated Kubernetes Cluster

Your cluster is now up and running inside its own host-only network—completely isolated from your Mac and the rest of your home network. But can you bridge the gap and manage your cluster directly from macOS?

Right now, kubectl only works inside the control plane VM. Your mission: configure secure connectivity so you can run kubectl from your Mac, just like you would with any remote cluster.

Hints to get started:

You’ll need to copy the admin.conf kubeconfig file from your control plane VM to your Mac.
The Kubernetes API server (kube-apiserver) is only accessible from inside the host-only network. Consider using SSH port forwarding, a VPN tunnel or setting up the required internal networking routes to expose it to your Mac.
Make sure your Mac can reach the control plane node’s IP (e.g., 192.168.100.2:6443), but don’t expose it to your entire home network for security reasons.

Can you set up seamless, secure access? Give it a try and unlock full control of your cluster from your Mac!

Troubleshooting

Node stuck on NotReady after reboot / Persistent swap issues

Restarted your worker node and it’s status stays stuck on NotReady? It could be that you still have swap enabled on your machine(s). If you followed our steps before, you probably have swap disabled in the usual locations but it doesn’t hurt to check!

Have a look into at the Kubelet logs with journalctl -n 10 -u kubelet --no-pager. At the bottom we see the error running with swap on is not supported, please disable swap or set –fail-swap-on flag to false. This means we still have swapping enabled on our system. First and foremost, temporarily disable swap with sudo swapoff -a.

We need to make sure all of the swap entries in /etc/fstab are commented out. At the very bottom, there is a line with /swap.img, we need to make sure this line is commented out. Either manually edit the line to #/swap.img or run

`1`	`sudo sed -i '/\/swap.img/ s/^/#/' /etc/fstab`

Let’s check if there’s any systemd units left that enable swapping systemctl list-units --type swap. This machine has a systemd unit that enables swapping, meaning that after a reboot swapping will be enabled again even after disabling it in /etc/fstab. Copy the name of the UNIT and run the following command, changing swap.img.swap if it differentiates from your unit name.

`1`	`systemctl mask swap.img.swap`

Restart your machine, repeat the journalctl command for the kubelet. The swap errors have disappeared! Our kubelet is running healthy again!

Containerd CRI Plugin Disabled

The CRI plugin is necessary for the kubelet to interact with containerd. During the installation of containerd it’s possible that the CRI plugin get’s added to the disabled_plugins list in containerd, let’s check. As we can see, cri is in my disabled_plugins list. Run the following command to remove CRI from the disabled plugin list and restart containerd.

1
2

sed -i '/disabled_plugins/s/\["cri"\]/[]/' /etc/containerd/config.toml && \
systemctl restart containerd

Advanced User Data Collection: Performance, Precision, and Persistence

Tue, 06 May 2025 12:28:22 +0100

We’ve arrived at the third and final iteration of the three part marketing blog series. In the first post we looked at how data is collected and how users are being tracked on the web. The second instalment focussed on common practices to gather deeper insights from your data. Now, in this final part, we’ll build on those practices and explore advanced strategies; optimizing data collection, enhancing tracking accuracy, and leveraging server-side techniques to maximize marketing performance.

Some of the techniques outlined in this post are in an ethical grey area. Precisely, server side tagging and cname cloaking can get around ad-blocking filters your users may use. These practices draw sharp criticism from privacy enthousiasts and activists alike, however, this does not stop businesses from implementing them. At Qstars IT we do not endorse these techniques, this post is for educational purposes only.

Server Side Tagging (SST)

You may have heard this before; “Amazon found that every 100 milliseconds of added latency costs them 1% in sales”. Amazon’s claim is widely cited, but it’s not the only proof that site speed has a real economic impact. Google, Akamai and Portent have all. released studies showing that marginal gains in site speed can increase conversion rates, user traffic and customer satisfaction.

In our last post, we looked at adding tracking code via tag managers. As we learned, these tags are individual snippets of code that get executed on the user’s browser which measure a variety of statistics about the user’s site visit. In a traditional setup, every action is executed on the visitors device. The computation required to gather the statistics, format and/or manipulate data and even the reporting to analytics services (such as Google Analytics) is done by the user.

This is not ideal, as you can imagine there are many downsides to the traditional approach. What if your end user(s) have filters that deny their browser from talking to Google Analytics? That’d mean you aren’t actually receiving the metrics that your tracking installation has collected. Or your visitors are on an older device? A heap of tracking snippets can gradually start slowing down the browsing experience for your users, especially folks on older hardware will get hit hardest by these performance penalties. And is it even a good idea for such important business logic to depend on the varying and unstable environments that are your customer’s browsers?

Technical Improvements

This is where Server-Side Tagging (SST) comes in. As the name suggests, SST is used to decouple the business logic from our data collection. By deploying SST, you ensure that your end users’ devices are only responsible for collecting data, while any transformations, validation, or reporting is handled by your central machine.

In addition to increased reliability of the base components (e.g., data collection through JavaScript execution), there are also fewer network requests, which can significantly improve performance—especially for visitors with a slower internet connection.

SST also implements smaller optimizations, such as more efficient batch sending of raw event data to your backend.

Furthermore, SST enhances security by leveraging HttpOnly cookies, reducing exposure to client-side attacks. HttpOnly cookies are cookies that can’t be accessed by JavaScript, reducing the risk of XSS attacks. They are sent with HTTP requests, protecting sensitive data like session IDs from theft.

Analytics Improvements

You also gain greater control over data handling, improving accuracy, regulatory compliance and third party integrations.

Improved data accuracy

As SST processes events on dedicated server than your user’s device, it reduces inconsistencies caused by your visitor’s environment. Network issues, browser restrictions (adblockers) and client-side execution failures can lead you to incomplete event tracking.

Regulatory compliance and governance

Privacy regulations like GDPR, CCPA, and ePrivacy Directive impose strict guidelines on data collection and user consent. With SST, companies can better manage data anonymization, user consent enforcement, and data retention policies before forwarding data to external analytics providers. By processing data in a controlled environment, organizations can ensure compliance without relying entirely on third-party scripts.

Third party integrations

Many marketing platforms and analytics tools require event data in a specific format. With SST, businesses can preprocess and normalize data before sending it to Google Analytics, Facebook Pixel, or other third-party services. This leads to cleaner, more structured reporting and ensures that the data is optimized to each platform’s requirements.

Cross-device tracking

SST allows for server-managed user identifiers, which improves attribution models by connecting data points across different sessions and devices, leading to a more comprehensive view of the customer journey.

CNAME Cloaking

CNAME cloaking is a technique used to bypass content and ad-blockers by masking third-party tracking scripts under a first-party domain. Essentially, it makes third-party requests appear as if they’re coming from your own domain, allowing them to slip past filters that would otherwise block or restrict them.

Let’s break it down: a CNAME record is a type of DNS record that allows one domain to alias another. In this case, you’d configure something like track.yoursite.com to point to a third-party tracking service (e.g., a CDN or analytics platform). From the browser’s point of view, everything appears to originate from your domain—even though the underlying infrastructure is external.

This tactic is increasingly used to maintain visibility in analytics and marketing platforms, despite widespread use of privacy tools and extensions that block known trackers.

Why it works

Most ad-blockers use predefined lists (like EasyPrivacy or uBlock Origin filters) to identify and block known third-party domains. But when you serve those same scripts through your own subdomain via a CNAME, they appear “first-party” and bypass detection. Since the blocklists rely on domain name matching, this trick sidesteps their protection mechanisms.

This gives businesses continued access to event data and user behavior metrics that would otherwise be lost, which can significantly impact attribution models and campaign optimization.

Controversy and ethics

CNAME cloaking is controversial. It effectively disguises third-party scripts, making it harder for users to detect tracking and potentially circumventing their expressed preferences (like Do Not Track or blocking third-party cookies). This practice has been called deceptive by privacy advocates and regulators.

In fact, the French CNIL (Commission Nationale de l’Informatique et des Libertés) has explicitly warned against the misuse of CNAME cloaking, stating that it can mislead users and obscure data flows—an issue particularly sensitive under GDPR.

Some browsers and privacy-focused organizations have started taking action. Firefox, for instance, has implemented mechanisms that detect and block cloaked CNAME trackers.

⚠️ At Qstars IT, we acknowledge the utility of this technique in maintaining analytics integrity, but we also stress the importance of ethical marketing practices and compliance with evolving privacy regulations.

Wrapping it up

Over the course of this three-part series, we’ve taken a deep dive into the mechanics of modern digital advertising. From understanding how user data is collected and processed to refining marketing strategies through smarter tracking and advanced technical solutions. As the landscape continues to evolve, businesses face an ongoing balancing act between optimizing ad campaign performance and respecting user privacy. By staying informed, making conscious decisions, and choosing the right tools for the job, marketers can unlock the full potential of their data while building trust with their audience.

Optimizing marketing efforts with user data

Thu, 06 Feb 2025 09:18:00 +0200

Note: As developers of QStars, we are interested in this technology from a technical and research perspective. However, we do not provide any services related to martech or digital advertising. Our focus remains on understanding and exploring these technologies rather than offering them as a product or service.

This blog post is the second installment of a three part series. Today we delve deeper into more technical details of digital advertising and cover the methods to collect data from your users.

Why collect user data?

User data is invaluable for making informed business decisions. By analyzing user interactions on your website or application you are able to identify trends, optimize conversion rates and personalize marketing efforts. Say there is a disproportionate amount of people viewing a product but not purchasing it; you can experiment with different messaging strategies or add social proofs and measure how these changes impact your conversion rates.

Tag Management Systems

As we touched on briefly in the previous blog post, tag managers play a critical role by allowing you to quickly create, update and deploy bundles of code on your website. These systems simplify data collection by having one-click presets and integrations in addition to allowing you to ship your own custom code.

Tag managers usually employ a Data Layer on your website. This javascript object is hidden from end users but acts as a middleman between your website and various analytic tools (such as Google Analytics). Think of the data layer as a centralized repository where all the data your site collects about a user gets stored, making the collection structured and accessible.

Here’s an example of a data layer for tracking an e-commerce purchase.

window.dataLayer = {
  event: "purchase",
  ecommerce: {
    transaction_id: "ORD12345",
    value: 150.0,
    currency: "USD",
    items: [
      {
        item_id: "prod_001",
        item_name: "Wireless Mouse",
        price: 50.0,
        quantity: 1,
      },
      {
        item_id: "prod_002",
        item_name: "Mechanical Keyboard",
        price: 100.0,
        quantity: 1,
      },
    ],
  },
};

With a data layer you are standardizing user behaviour data, making it much easier for your marketing teams to gain insights from it.

Privacy regulation like the GDPR and CCPA have significantly altered how a business is allowed to collect and store data about its users. The biggest requirement is having to obtain explicit consent before deploying any tracking cookies or scripts.

Without this consent businesses must only rely on anonymized data, limiting their ability to uniquely identify users and serve them personalized ads. This can also weaken segmentation efforts, making it harder to understand user behavior and preferences.

To address this challenge, many businesses implement cookie consent management platforms (CMPs). These platforms provide users with options to control how much they are tracked while ensuring businesses remain compliant with local regulations. Some CMPs also enable partial tracking capabilities, where anonymized data is collected if full consent is not granted.

A/B Testing

A/B testing is the building block of data-driven working and optimizing your digital marketing efforts. Also known as split testing, it is a method used to compare two or more variations of a webpage, advertisement, or email campaign to determine which one performs better.

By leveraging data collected through tag management systems and analytics tools, businesses can autonomously test elements such as headlines, call-to-action buttons, images, and pricing strategies. This process involves showing different versions of a page or ad to different segments of users and analyzing their interactions to identify the most effective design, messaging, or layout.

The key benefits of A/B testing are:

Data-driven decision making eliminates the guesswork and validating user preferences with real data.
Improving conversion rates by identifying the most effective ad copy, design and product pricing variations.

Tag managers allow you to quickly implement A/B tests on your site with the added benefit of it being stored in your data layer. This way businesses can seamlessly analyze results in their analytics platform without any change in your reporting tools.

Predictive Analytics & AI

Beyond A/B tests, businesses can leverage the power of predictive analytics and machine learning to anticipate user behaviour.

Although setting up custom predictive analytics models requires advanced technical expertise, these models can help with:

User behaviour forecasting: By analyzing historical data, AI models try to predict what users are most likely to convert
Dynamic creative optimization: Dynamically adjusts website content, product recommendations, and ad creatives based on user patterns
Fraud detection & risk analysis: Identify patterns that indicate fraudulent activity or purchasing.

Platforms like Google- & Meta Ads heavily rely on machine learning algorithms to enhance their ad delivery targeting, making it crucial that you are sending them data directly collected from your user’s behaviour. Providing high-quality data to these advertising managers significantly increases return on ad spend (ROAS), cost-per-acquisition (CPA) and optimize ad bidding in real-time.

Up next

In today’s post, we explored how businesses can enhance their digital advertising efforts by integrating user data. We covered the importance of tag managers, cookie consent, split testing, and the power of AI.

In the final post of this series, we’ll dive into strategies for maximizing the data you collect from users. We’ll also discuss approaches to handle ad blockers and other methods to minimize the impact of tracking tools on your website’s performance.

Understanding digital data collection

Fri, 27 Dec 2024 09:38:00 +0200

This blog post marks the beginning of a three-part series exploring the digital advertising landscape. In today’s post, we’ll examine the data collection mechanisms employed by ad platforms to deliver personalized advertisements. Future posts will delve into practical strategies and examples to help optimize digital advertising efforts, which are based on the concepts we’ll familiarize ourselves with today.

The current landscape

The global marketing industry is, and has been, booming. While estimates vary, analysts predict that the market could reach anywhere between $700 billion and $1.2 trillion by 2028. To put that into perspective, the digital advertising market alone is on track to surpass the GDP of Belgium. That’s an immense amount of purchasing power.

A major driver behind this growth is the proliferation of personalized advertisements. Google leads the charge in the ad ecosystem, leveraging their dominance in search engines, browsers and platforms like YouTube to collect enormous amounts of user data. As they have been accused of deploying questionable tactics to collect this data, like gathering user data in Incognito mode through their Chrome browser.

So this begs the question: where does all of this actionable data come from?

How data is collected

The phrase “If something is free, you are the product” perfectly encapsulates the reality of digital advertising. There is a growing regulatory push around the globe to protect consumer privacy and control how companies store, track and handle personal data. While cookie acceptance banners have become inescapable, companies already employ more key tactics to monitor your digital footprint.

1. Cookies: The Foundation

Cookies are small text files that the browser stores on your device on behalf of the websites you visit. These cookies can be placed in your browser by different vendors, which introduces the concept of first vs third party cookies.

First party cookies are explicitly set by the website you are visiting. They are commonly used for the site to remember your shopping cart or login status. They can also be used to track your behaviour across the website.

Third party cookies are created by external parties. When you browse a site (and consent to their privacy policy), code is loaded onto your device from advertising networks and social media platforms. This enables them to store an identifier on your device which they can read when you visit another site that deploys code from this vendor.

Google Analytics is a common third party cookie vendor that tracks user behaviour across its network of websites. Its tracking code sets cookies to collect data like page views and interactions, sending it to Google’s servers. If the user visits another site using Google Analytics, their browsing habits can be linked, helping businesses refine marketing and deliver targeted ads.

2. Fingerprinting

Fingerprinting is a tracking method that’s harder to detect and block. It aims to identify and track users across websites and in different sessions.

It works by collecting a variety of data points about a user’s device, browser, and geolocation. While there are virtually endless points that can be tracked, some common ones are:

Browser and device:
- Type
- Version
- Screen size
- Language settings
- Timezone
- Operating system
- Graphics processing unit (gpu) & processor details
Your IP address and the approximate location it resolves to.
Any installed browser add-ons or tools.

By combining all of these factors, fingerprinting creates a unique identifier that can persist even if you deny tracking or clear your cookies.

3. Tag Managers

Tag managers allow you to efficiently deploy small bundles of code, referred to as “tags”. These tags are triggered on your website by a specific user event, such as clicking on a product or viewing a page. They have become popular for streamlining the process of integrating user tracking on websites.

One of the most popular tag management tools is Google Tag Manager, though similar solutions are offered by Adobe, Tealium, and others.

Tag managers integrate seamlessly with analytics tools, data warehouses and advertising platforms. This has made it easier to get actionable data from tracked user behaviour. Within minutes you can segment audiences and identify high-value customers for retargeting.

What’s next

Today we’ve discussed three primary methods of data collection: cookies, fingerprinting, and tag managers. As the digital advertising industry continues to expand, consumer privacy and government regulation is becoming a major concern. Future posts in this series will dive deeper into strategies for optimizing digital advertising efforts while staying compliant with privacy regulations.