My K3S infrastructure: architecture and technical choices

Overview

My infrastructure runs on a K3S cluster made up of two nodes connected through Tailscale:

The whole thing is managed with pure GitOps: all the configuration lives in a GitHub repo, and a push to main automatically triggers a deployment.

No Terraform, no Ansible, no provisioning scripts. Just Kubernetes YAML files managed by ArgoCD.

Repo : github.com/BaptTF/vps-infra

Note on the evolution: historically, the control-plane was running on Contabo and a second node (bapt-debian, a VM at a friend’s place with 8 GB) was acting as the worker. When this VM went down, I looked for a replacement and found Hetzner. I swapped the roles: Hetzner became the control-plane (it has more capacity and is better suited to critical workloads), and Contabo became the worker.

GitOps with ArgoCD

I use the App of Apps pattern: a root-app.yaml points to an apps/ folder containing the Application CRs for each service. Each app is configured with autoSync, prune: true, and selfHeal: true. If I accidentally change something by hand on the cluster, ArgoCD automatically puts it back into the state defined by Git, and if I remove something from the repo, it gets deleted (you need to be careful with PVCs with this configuration).

Automatic updates

Renovate watches the versions of the Helm charts and creates PRs automatically. ArgoCD Image Updater is only used to update my own images automatically; it commits directly to the repo to stay 100% GitOps.

To enable or disable a service, I just move its file between apps/ and disable-apps/. That’s all.

Network

Public ingress: Traefik

Traefik acts as the ingress controller for all public traffic. It listens on ports 80, 443, and 22 (historically for Forgejo SSH, though I don’t use it anymore). Routing is handled through IngressRoute CRs toward *.bapttf.com.

The TLS certificates are managed by cert-manager with a DNS-01 challenge through the Cloudflare API. This makes it easy to get certificates without having to run HTTP challenges.

Mesh VPN: Tailscale

Tailscale is at the heart of the architecture. The two K3S nodes communicate through Tailscale’s WireGuard tunnel, and Flannel encapsulates its VXLAN overlay inside that tunnel. Some services like Grafana, Bifrost, or LLDAP are only exposed on the Tailnet through the tailscale ingress class. For administration, kubectl and the ArgoCD UI are only accessible through Tailscale.

Cloudflare

At the front, Cloudflare provides the default WAF, DDoS protection, and DNS management. The Cloudflare API is used by cert-manager for the DNS-01 challenges. The only downside is that I can’t have sub-subdomains (*.*.bapttf.com) with Cloudflare’s free plan, but it’s already really good; it’s just not super clear.

Authentication

The auth stack is LLDAP for the LDAP directory (with a web UI to manage users and groups) and Authelia for the SSO portal. Authelia provides ForwardAuth through a Traefik middleware to protect routes, and an OIDC provider for apps that support it, such as Immich or the Hermes dashboards.

The access policies are based on LDAP groups: some services are accessible to the family, while others are only available to admins.

Secret Management

I use a two-tier model:

1. Sealed Secrets (Bitnami). Used only to bootstrap the first secret: the credentials to access Infisical. The sealing certificate is versioned in the repo.

2. Infisical. An external SaaS that manages all the other secrets. The Kubernetes operator automatically synchronizes the secrets from Infisical into Kubernetes Secrets, organized by path (/argocd, /cloudflare, /tailscale, /agents/bifrost, etc.).

Why Infisical?

When I looked into it, it seemed simpler than Vault (HashiCorp), which is massive for a small cluster, and more integrated than SOPS (which requires managing keys, encrypting and decrypting for every change, and makes secret rotation difficult). With Infisical, I have a web UI, rotation, environments, and a native Kubernetes operator. Was it the best choice? I don’t know. But it’s okay.

Databases

I use CloudNative-PG (CNPG) as the PostgreSQL operator. It manages a main cluster with 9 databases (ArgoCD, Authelia, Vaultwarden, Immich server, agents, TripKit, etc.) and a dedicated Immich cluster with the vectorchord extension for image similarity search.

Storage

Local

K3S’s default provisioner (local-path) is used for most PVCs: databases, config, and small volumes.

Hetzner Storage Box (SMB)

For Immich, photos and videos are stored on a Hetzner Storage Box mounted through the CSI SMB driver. It’s an economical choice: 1 TB of network storage for €3/month, much cheaper than increasing the VPS disk.

The Immich storage architecture:

• Local PVC: thumbnails, profiles, generated data
• Storage Box: uploads, main library, encoded videos, family photos

A FileBrowser is also mounted on this Storage Box to let the family manage its files.

Backup

Approach:

Deployed services

Active

Disabled (in standby)

Monitoring (kube-prometheus-stack), Forgejo, OpenWebUI, MinIO, Obsidian LiveSync, Velero, and a few others.

Retrospective

Good choices

• ArgoCD + GitOps: never SSH into a server to deploy again. It’s just great.
• Tailscale simplifies everything. Inter-node network, admin access, private ingress, it’s the same tool for everything.
• CloudNative-PG is managed PostgreSQL on Kubernetes, with automatic backups. It makes database management on Kubernetes really easy.
• Traefik with the IngressRoute CRs and cert-manager makes it really easy to expose services on a URL.
• Renovate + Image Updater mean updates are one click away in a PR, or automatic otherwise.

Evolution of the infrastructure

Historically, the control-plane was running on Contabo (8 GB), and a second node (bapt-debian, a VM at a friend’s place with 8 GB) was acting as the worker. When this VM went down, I had to manually scale several workloads down to zero on Contabo to avoid an OOM. I then looked for a new VPS to replace the worker. That’s Hetzner (15 GB). Along the way, I swapped the roles: Hetzner became the control-plane (it has enough RAM to support etcd and the critical workloads), and Contabo became a worker. In August, I’m going to upgrade Contabo from 8 GB to 12 GB of RAM. Not by choice: Contabo raised its prices and offers the upgrade for free in return.

Debatable choices

• Infisical creates a dependency on an external SaaS for secrets. I had hesitated between it and HashiCorp Vault, but Vault was overkill for what I needed. When I asked Gemini about it, it gave me this analogy: “Vault on an 8 GB VPS is like installing a Swiss safe to keep the keys to a Twingo.” Infisical is lighter, simpler to set up, and does exactly what I need without Vault’s complexity. Maybe I’ll switch in the future, but for now it’s working really well.
• No permanent monitoring. kube-prometheus-stack is disabled because I needed to save RAM. I need to re-enable my monitoring stack.

What I would change

• Keep monitoring active permanently on Contabo (worker, 12 GB), which has enough storage and memory headroom to host it.
• Automate the Restic backups, which are currently manual. I’m looking at sending them either to an S3 bucket, another Storage Box, or a NAS; I haven’t decided yet.