How Headscale coordinates your mesh without Tailscale’s servers

I’ve been running Tailscale in my homelab for about two years now. It works. The apps connect, the latency is fine, and I stopped thinking about it months ago, which is exactly what you want from infrastructure. But I kept wondering what was actually happening on the other end of that connection—what Tailscale’s servers were doing that made the whole thing tick. So I set up Headscale one weekend and spent a few hours reading through the code. What I found was simpler than I expected, and stranger in some ways too.

The thing Tailscale doesn’t advertise

Most VPN documentation makes it sound like your traffic flows through a central server. Tailscale doesn’t work that way. After the initial handshake, your traffic goes point-to-point. The control server—whether it’s Tailscale’s or your own Headscale instance—only does coordination. It tells your devices about each other. It doesn’t touch your actual data.

That separation is the whole reason Headscale can exist at all. Tailscale open-sourced the client code years ago. The control server was closed, but someone reverse-engineered the protocol well enough that Juan Font could write a compatible replacement. It’s not perfect compatibility—we’ll get to that—but it’s close enough to be useful.

Here’s what I mean by coordination: your laptop joins the network. Headscale registers it, assigns it a magic IP address from the 100.64.0.0/10 range. Then Headscale tells your phone, your server, your other laptop, and anything else that’s connected that this new device exists and where to find it. Each device stores that information and tries to establish a direct connection using WireGuard.

The WireGuard handshake and key distribution

When you first connect a device to Headscale, the Tailscale client on that device generates a WireGuard keypair. It sends the public key to Headscale over HTTPS. Headscale stores that public key alongside the device’s name, user, and a handful of metadata.

Every time another device connects, or every time Headscale’s node list changes, it rebuilds a map file for each peer. This file contains the WireGuard public key of every device the peer should know about, plus the last known IP addresses where it might be reachable. Headscale pushes these updates to clients regularly, and clients subscribe to changes using a gRPC stream.

This is where things get a little weird, actually. The gRPC connection stays open. Your client is basically listening for updates from Headscale. When you add a new device, Headscale pushes a notification down that open connection. The client wakes up, learns about the new node, and tries to reach it. If the new device is on the same network, they’ll connect directly. If they’re across the internet, they attempt a NAT traversal technique called STUN (Session Traversal Utilities for NAT). If that fails, they fall back to a DERP relay.

DERP relays and the messy middle ground

DERP—Distributed Ephemeral Relay Protocol—is Tailscale’s own invention. It’s a simple UDP relay. If two devices can’t reach each other directly, they both connect to a DERP relay server, and that relay moves packets between them. It’s not as efficient as a direct connection, but it works through almost any firewall.

Here’s the catch: Headscale can use public DERP servers, or you can run your own. Most homelabbers just use Tailscale’s public ones, which means some of your data is still touching Tailscale infrastructure, even though you’re self-hosting the control plane. That bothered me enough that I spun up a DERP server, which is its own little engineering decision. The relay is fast—it mostly just copies bytes around—but you need at least one instance accessible from the internet on a stable IP or hostname. I’m using a tiny cloud VM, which defeats some of the self-hosting purity, but it’s fifteen bucks a month and I was already renting it for other things.

The other option is to accept that some connections will fail without a relay, or to engineer your network so that doesn’t happen. In a homelab scenario, you probably have decent internet from your main location, so the tradeoff is acceptable. I just wanted to know what I was trading away.

The state Headscale keeps and why it matters

Headscale stores everything in a database. By default, it’s SQLite, but it supports PostgreSQL. The schema is straightforward: users, nodes, keys, routes, access control lists. When I first read through the database migration files, I was struck by how sparse it is. Not in a bad way—just efficient. No bloat.

A few fields matter if you’re troubleshooting. Each node has a key (WireGuard public key), an ip_address (the 100.x magic IP), a last_seen timestamp, and a hostname. When you list nodes, you’re just querying this table. When a client connects to the API, it authenticates using a pre-shared key that you generate and distribute. No users, no passwords. Just API keys. Simpler than Tailscale’s auth flow, actually.

The access control is where it gets interesting. Headscale supports Tailscale ACLs, which are defined in a JSON or HuJSON file. You specify which users or tags can talk to which other users or tags. Here’s a snippet from mine:

{
  "acls": [
    {
      "action": "accept",
      "src": ["tag:homelab"],
      "dst": ["tag:homelab:*"]
    },
    {
      "action": "accept",
      "src": ["tag:mobile"],
      "dst": ["tag:homelab:*"]
    }
  ]
}

When you try to connect from your phone to a server, Headscale checks the ACL. If it doesn’t match, the connection fails at the WireGuard level—Headscale refuses to tell your phone about that node. It’s not a firewall rule; it’s network topology as policy. That’s elegant, but it also means you have to get the ACL right or you’ll debug for an hour wondering why your device isn’t even aware a server exists.

The gRPC API and the real bottleneck

All of this runs on gRPC. Headscale exposes a gRPC API that the Tailscale clients call. The API definition is in protobuf, which means it’s language-agnostic and pretty efficient. The main endpoints are:

• RegisterNode: device checks in, Headscale assigns it a magic IP
• GetNode: fetch info about a specific node
• ListNodes: get all nodes you have permission to see
• StreamMapResponse: the open subscription I mentioned, where updates arrive

I ran a single Headscale instance on a $5 Hetzner cloud server for six months. It handled my homelab fine. About fifteen devices, mostly idle. The CPU was barely visible. Memory usage hovered around 80MB. The bottleneck was network—not Headscale’s network, but the WireGuard tunnels themselves. If you’re pushing a gigabyte of data through a relay, well, that’s just physics.

One thing I didn’t expect: Headscale is stateless enough that you could run multiple instances behind a load balancer, as long as they point to the same database. The gRPC connections themselves are stateful, but the protocol is designed so a client can reconnect and pick up where it left off. I haven’t tried it, but it’s nice to know the option exists if you wanted real redundancy.

The compatibility problem that might bite you

Headscale implements most of Tailscale’s control plane protocol, but not all. The parts it doesn’t implement are mostly the fringe stuff: Tailscale’s advanced DNS features, some of the subnet routing complexity, and a few newer ACL features that shipped after Headscale’s last major release.

I hit this with exit nodes. I wanted one device to be an exit node, to act as a VPN gateway. The Tailscale documentation shows how to enable it. On Headscale, the flag exists, the client supports it, but the control server doesn’t do anything with it. The exit node advertises itself, but the ACLs don’t respect that configuration. You have to manually write ACL rules to make it work, which is fine, but it’s not transparent.

It’s not a deal-breaker. It’s just a place where the abstraction leaks. You’re no longer just using Tailscale with your own server; you’re using Tailscale-ish with your own server, and you have to know where the cracks are.

Docker, deployments, and what I actually run

Most people run Headscale in Docker. There’s an official image. Here’s roughly what mine looks like:

services:
  headscale:
    image: headscale/headscale:v0.23.0
    ports:
      - "8080:8080"
    volumes:
      - ./config:/etc/headscale
      - ./data:/var/lib/headscale
    environment:
      - TZ=UTC
    restart: unless-stopped

I use Traefik in front of it for HTTPS and routing. The config file is YAML, and it’s pretty readable. Port 8080 is the gRPC API. You can also expose a separate HTTP port for the web UI, though the web UI is more for admin tasks—listing nodes, generating auth keys, viewing logs—rather than something users interact with directly.

One small thing that annoyed me: Headscale doesn’t auto-reload the config. If you change the ACL file or the config YAML, you have to restart the container. It’s not a big deal in a homelab, but it means zero-downtime policy updates are off the table unless you run multiple instances with a load balancer. I just accept the brief disconnect.

Why this matters, and what it means for you

The reason to understand Headscale’s architecture isn’t to run it perfectly. It’s to know what you’re giving up and what you’re gaining. You get full control over user management, ACLs, and where your metadata lives. You lose some Tailscale features and you have to maintain another service. The equation doesn’t work for everyone.

For me, the appeal was the independence. I don’t trust third parties with network topology. I trust them with infrastructure—I happily run on cloud VMs—but the map of how my devices talk to each other should be something I control. Headscale lets me have that without much complexity.

The architecture itself is pretty clean. The separation between data plane and control plane is clear. The state management is minimal. The dependency on gRPC is good—it’s been battle-tested and it’s efficient. The main architectural decision that matters is whether you run your own DERP relay or accept Tailscale’s. Everything else flows from that.

If you do set this up, expect to spend an afternoon with the documentation, especially on the ACL side. It’s not hard, just requires precision. And maybe keep a terminal open with headscale nodes list ready to debug when something’s not connecting the way you expect. That command will probably be your most-used tool.

Explore Headscale in our AI Homelab Toolkit.