I started running Tabby about four months ago because I got tired of my VS Code setup feeling like it was constantly phoning home. The Copilot telemetry logs were thick enough that I knew something was being tracked, and I work on stuff I’d rather keep private. GitHub’s pricing wasn’t even the main irritant — it was the principle. So I spun up Tabby on my homelab box with a spare RTX 3060 and haven’t looked back. Here’s the exact setup I’m running in production.
The Problem That Led Here
The real friction point wasn’t actually code completion speed or quality. It was control. I wanted my models, my context, my entire inference pipeline running on hardware I owned, not calling out to Redmond or San Francisco on every keystroke. Copilot works fine, but I had no way to audit what it was sending, how long it kept it, or which training data it touched. With Tabby, the model runs locally. Your code stays in your VPC or, in my case, your basement.
The second pain point was context. Out-of-the-box Tabby is good, but pointing it at your actual repository and letting it understand your codebase structure — that’s where it gets useful. The config I’m sharing does that.
Hardware and Prerequisites
I’m running this on an Intel i7-9700K with 32GB RAM and the RTX 3060 I mentioned. Tabby will work without a GPU, but you’ll feel the latency immediately. If you’re thinking about self-hosting code completion, assume you want some acceleration. The 3060 gives me solid inference speed for both the completion model (CodeLlama 7B) and the embedding model (BGE-small) without melting my electricity bill.
You’ll need Docker and Docker Compose, obviously. I’m on Docker 24.0.6 and Compose 2.20. Traefik sits in front as the reverse proxy, which isn’t strictly required, but I’m using it for auth and HTTPS, so it’s worth the three extra services in the stack.
One thing that surprised me: Tabby’s model download on first run is about 7GB depending on which models you choose. That’s not huge, but if you’re on a slow uplink, plan accordingly.
Docker Compose Configuration
Here’s what’s actually running:
version: '3.8'
services:
tabby:
image: tabbyml/tabby:0.9.4
container_name: tabby
restart: unless-stopped
ports:
- "8080:8080"
environment:
TABBY_DOWNLOAD_HOST: "huggingface.co"
RUST_LOG: "info"
volumes:
- ./tabby-data:/data
- /dev/nvidia0:/dev/nvidia0
- /dev/nvidiactl:/dev/nvidiactl
runtime: nvidia
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
networks:
- traefik-network
labels:
- "traefik.enable=true"
- "traefik.http.routers.tabby.rule=Host(`tabby.lab.local`)"
- "traefik.http.routers.tabby.entrypoints=websecure"
- "traefik.http.routers.tabby.tls=true"
- "traefik.http.routers.tabby.service=tabby"
- "traefik.http.routers.tabby.middlewares=auth@file"
- "traefik.http.services.tabby.loadbalancer.server.port=8080"
traefik:
image: traefik:v2.10
container_name: traefik
restart: unless-stopped
ports:
- "80:80"
- "443:443"
environment:
TRAEFIK_API_INSECURE: "false"
TRAEFIK_PROVIDERS_FILE_DIRECTORY: "/config"
TRAEFIK_PROVIDERS_FILE_WATCH: "true"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik/config:/config
- ./traefik/certs:/certs
networks:
- traefik-network
command:
- "--api.dashboard=true"
- "--providers.docker=true"
- "--providers.docker.network=traefik-network"
- "--providers.file.directory=/config"
- "--entrypoints.web.address=:80"
- "--entrypoints.websecure.address=:443"
- "--entrypoints.web.http.redirections.entrypoint.to=websecure"
- "--entrypoints.web.http.redirections.entrypoint.scheme=https"
networks:
traefik-network:
driver: bridge
volumes:
tabby-data:
The key lines here: the runtime: nvidia and the GPU device mapping. Without those, Tabby runs on CPU and you’re looking at 5-10 second latencies on completions. With GPU acceleration enabled, I’m seeing sub-second responses. The volume mount at /data is where Tabby stores downloaded models and its config, so persist that or you’ll re-download everything on container restart.
I pinned the image to 0.9.4 because newer versions had a regression in embedding model loading that I haven’t had time to troubleshoot. Real-world homelab work: you pin versions when things work and move forward only when necessary.
Traefik Configuration and Authentication
The docker-compose labels are routing traffic through Traefik, but you need the auth middleware defined outside the compose file. Here’s the separate config file:
// traefik/config/middleware.yml
http:
middlewares:
auth:
basicAuth:
users:
- "mustafa:$apr1$xK4J5.gW$5BL2U/7..." # htpasswd generated
Generate that password hash with htpasswd:
htpasswd -c .htpasswd mustafa
# then cat .htpasswd and paste the result into the YAML above
I’m using basic auth because Tabby’s native auth is new and still rough around the edges. Basic auth keeps random people from hammering my inference server, and combined with HTTPS (which Traefik handles with self-signed certs in my lab), it’s sufficient for a homelab where no real attackers are going to scan my internal DNS.
Environment Variables and Model Selection
The critical part most people miss: you don’t configure model selection in the compose file. You do it on first run. When Tabby starts, it downloads a default model (CodeLlama 7B Instruct by default). If you want something different, you have two paths: edit the config after the container settles, or pass environment overrides. I ended up using neither, actually. The defaults were good enough that I left them alone.
The TABBY_DOWNLOAD_HOST environment variable matters if you’re behind a restrictive network or want to use a local Hugging Face mirror. I’m pointing at the public Hugging Face CDN and it works fine from my location. The RUST_LOG set to info gives me readable logs without the debug noise.
First Run and Model Download
Fire up the stack:
docker-compose up -d
Watch the logs:
docker-compose logs -f tabby
The first startup takes a while. Tabby is downloading the completion model, the embedding model, and initializing them. On my connection, this was about eight minutes. Once you see repeated log lines about “server listening on 0.0.0.0:8080”, it’s ready. Hit https://tabby.lab.local in your browser, log in with your htpasswd credentials, and you should see the Tabby dashboard.
One gotcha I hit: if you have NVIDIA drivers installed but not the container toolkit, you’ll get permission errors on /dev/nvidia0. Install nvidia-docker or use the proper device runtime. The error message is unhelpful (just says “permission denied”), so I’m mentioning it here explicitly.
Connecting VS Code and JetBrains
The Tabby extension for VS Code lives on the marketplace. Point it at https://tabby.lab.local and use the same htpasswd credentials you set up in Traefik. The extension handles the auth handshake. Once connected, you’ll see the Tabby icon in the status bar.
Completions appear as inline suggestions, familiar if you’ve used Copilot. The first few times you trigger one, you’ll notice a slight pause while the model thinks. After that, it usually feels snappy enough that you don’t mind reaching for it.
JetBrains IDEs (I tested on PyCharm 2023.2) have a native Tabby plugin that works nearly identically to the VS Code extension. Same endpoint configuration, same experience.
Repository Context and Configuration
This is where Tabby differs from generic LLM completion. You can point Tabby at your repository root and it indexes files, so context isn’t just the current buffer. Navigate to the Settings tab in the Tabby UI, click “Indexing,” and add your repo directories. The indexing runs asynchronously. For a medium-sized repo (50K files, maybe 2GB of code), indexing takes 15-20 minutes.
After indexing completes, completions that reference files outside your current buffer become much better. I was skeptical about this, but it’s legitimately useful. Tabby suggested a function signature from a completely different module based on context, and it was right.
The one limitation I’ve hit: if your repository changes significantly (lots of new files, deleted files, refactoring), you need to re-index. There’s no incremental indexing yet, so it’s all-or-nothing. For my use case, re-indexing once a week manually is fine. If you’re in a large team with constant churn, this might be annoying.
Monitoring and Resource Usage
GPU memory usage sits around 3-4GB under normal load with the 7B model. That leaves plenty of headroom on the 3060. CPU usage is minimal while waiting for requests, spiking to about 30-40% utilization when processing completions. RAM stays under 8GB on the Tabby container itself, though the host is using more for system buffer caches.
I’m not monitoring this obsessively, but I’ve checked nvidia-smi a few times and the memory allocation is clean. No memory leaks that I’ve noticed over the four months of continuous running.
The real cost is electricity. A 3060 under load draws about 160W. Tabby’s inference isn’t running constantly — mostly idle, spiking when you hit Tab. So the practical overhead is probably 20-30W average, depending on how much you’re using it. That’s negligible in the bigger picture of a homelab, but worth knowing.
What I’d Do Differently
If I were starting over, I might skip Traefik and use Nginx Proxy Manager instead, mostly because NPM’s UI for managing auth and HTTPS is a bit more straightforward. Traefik’s learning curve is steep for something this simple. But Traefik works, so I haven’t changed it.
I’d also keep better notes on model versions. I pinned 0.9.4 because something broke in 0.10, and now I can’t remember what. Next time I’ll document that in comments before it matters.
The indexing limitation is worth planning for. If you’re thinking about this for a team setup, Tabby might not be your tool yet. It’s solid for solo work or small teams where re-indexing weekly isn’t a burden. For anything larger, you probably want a SaaS solution or a setup with more sophisticated caching.
FAQ
Does Tabby require a GPU?
No, but you’ll regret not having one. CPU-only inference on a 7B model takes 5-10 seconds per completion. A basic GPU like the 3060 or even a 2060 brings that down to under a second, which is the difference between a tool feeling responsive and feeling like you’re waiting.
What’s the minimum RAM needed?
The Tabby container itself uses about 2GB at rest and 4-5GB under load. The embedding model adds another 1-2GB. So realistically, have 8GB available on the host. Less than that and you’ll hit swap, which defeats the purpose of local inference.
Can I use Tabby without Traefik?
Yes. Tabby exposes port 8080 directly. You can hit it at http://localhost:8080 without any reverse proxy. Traefik adds HTTPS and authentication, but if you’re only using Tabby on your local network or over VPN, plain HTTP is fine.
How often do I need to re-index my repository?
There’s no automatic re-indexing. If files change, the index goes stale. I re-index weekly as a maintenance task. For smaller repos or repos that change slowly, monthly is probably sufficient. There’s no fixed rule — you’ll feel it when the context gets out of sync.
Is Tabby as good as Copilot?
It’s different. Copilot has access to training data from billions of public repositories. Tabby has a 7B parameter model trained on code, which is capable but not as broad. Where Tabby wins is control, privacy, and repository context. For my specific codebases, Tabby’s suggestions are just as good. For random code patterns you haven’t written before, Copilot probably has an edge. Pick based on your priorities, not on raw quality alone.
Explore Tabby in our AI Homelab Toolkit.