Console Login

GitOps Workflows in 2023: From 'Works on My Machine' to Production Sovereignty

GitOps Best Practices: Infrastructure as Code meets Data Sovereignty

If you are still SSH-ing into servers to run docker-compose up or, god forbid, manually editing config files in /etc/, you are creating a ticking time bomb. In 2023, manual operations (ClickOps) are not just inefficient; they are a form of professional negligence.

I have spent the last decade cleaning up messes left by "quick fixes" applied directly to production servers. The only way to maintain sanity and stability is GitOps: using Git as the single source of truth for your declarative infrastructure and applications.

But here is the uncomfortable truth most tutorials ignore: GitOps tools are resource hogs. Running a reconciliation loop every 3 minutes requires consistent CPU cycles and fast I/O. If you throw a heavy ArgoCD instance on a budget VPS with noisy neighbors and spinning rust (HDD), your deployment pipeline will time out. I've seen it happen.

The Architecture: Pull vs. Push

Traditional CI/CD (Jenkins, GitLab CI) often relies on a "Push" model. The CI server has god-mode access to your production cluster and pushes changes. This is a security nightmare. If your CI server is compromised, your production environment is gone.

We use the "Pull" model (ArgoCD or Flux). The agent sits inside your cluster and pulls changes from the Git repo. It compares the desired state (Git) with the live state (K8s) and forces them to match.

The Hardware Requirement

When ArgoCD triggers a sync on a cluster with 50+ microservices, the API server gets hammered. Etcd writes spike. If your underlying storage latency is high, the reconciliation fails. This is why we default to CoolVDS instances. Their NVMe storage guarantees the IOPS needed for etcd to remain healthy during "sync storms."

Step 1: The Foundation (Infrastructure as Code)

Before deploying applications, you need the cluster. Don't build it manually. Use Terraform. Below is a stripped-down example of how we bootstrap a K8s node. Note the emphasis on swap management—Kubernetes schedulers hate swap.

resource "coolvds_instance" "k8s_worker" {
  image     = "debian-11"
  label     = "k8s-worker-oslo-01"
  region    = "no-oslo-1" # Keep it local for low latency
  ipv6      = true
  
  # Critical: NVMe ensures etcd latency remains <10ms
  disk_type = "nvme"

  user_data = <<-EOF
    #!/bin/bash
    # Disable swap immediately for Kubelet stability
    swapoff -a
    sed -i '/ swap / s/^/#/' /etc/fstab

    # Enable IP forwarding
    cat <<SYSCTL > /etc/sysctl.d/k8s.conf
    net.bridge.bridge-nf-call-ip6tables = 1
    net.bridge.bridge-nf-call-iptables = 1
    net.ipv4.ip_forward = 1
    SYSCTL
    sysctl --system
  EOF
}

Step 2: Implementing ArgoCD

Once your cluster is running on solid metal (or a high-performance VPS), install ArgoCD. Do not expose the ArgoCD UI dashboard to the public internet. Use port-forwarding or a secure ingress with strict IP allow-listing.

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Configuring the Application

Here is a declarative Application manifest. This file sits in your "management" repo. Notice the selfHeal policy. This is the magic. If a junior dev changes a limit manually on the server, ArgoCD detects the drift and reverts it instantly.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: production-api
  namespace: argocd
spec:
  project: default
  source:
    repoURL: 'git@github.com:your-org/infra-manifests.git'
    targetRevision: HEAD
    path: k8s/production/api
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true      # Deletes resources not in Git
      selfHeal: true   # Reverts manual changes
    syncOptions:
      - CreateNamespace=true

The Norwegian Context: Latency and Law

Why does location matter for GitOps? Two reasons: Schrems II and NIX (Norwegian Internet Exchange).

1. Legal Compliance: In the post-Schrems II world, relying on US-owned cloud providers for core infrastructure is legally risky for Norwegian businesses handling personal data. By hosting your Kubernetes nodes on CoolVDS (which operates under Norwegian jurisdiction), you simplify your GDPR compliance posture. Your state files and secrets remain within the EEA.

2. Latency: If your Git repository is hosted on a generic cloud but your servers are in Oslo, you are introducing unnecessary lag. But more importantly, if your users are in Norway, your managed hosting needs to be in Norway.

Pro Tip: Check your latency to the NIX. Run mtr --report nix.no from your current server. If you are seeing >15ms from within Norway, your provider is routing traffic inefficiently. On CoolVDS Oslo nodes, we typically see <2ms.

Optimizing the Network Stack

Default Linux kernel settings are tuned for general-purpose desktop usage, not high-throughput GitOps pipelines pulling gigabytes of container images. You need to tune the TCP stack. Add this to your /etc/sysctl.conf:

# Increase the read/write buffer sizes for high-speed replication
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216

# Protect against SYN flood attacks (essential for public facing APIs)
net.ipv4.tcp_syncookies = 1
# Reuse connections in TIME_WAIT state
net.ipv4.tcp_tw_reuse = 1

Apply these with sysctl -p.

The Trade-Offs

GitOps is not free. It adds complexity. You now have to manage the manager (ArgoCD). If ArgoCD goes down, you can't deploy. This creates a single point of failure.

To mitigate this:

  • Backup etcd: Your cluster state is your lifeblood. Snapshot it hourly.
  • Resource Requests: Give the ArgoCD repo-server component at least 1GB of RAM. It caches Git repos in memory.
  • DDoS Protection: Automated pipelines trigger frequent network connections. Ensure your provider (like CoolVDS) has intelligent DDoS filtering that doesn't flag your own CI/CD traffic as an attack.

Final Thoughts

Automation is binary: you either have it, or you don't. Half-automating a workflow is just manual work with extra steps. By pairing a strict ArgoCD workflow with the raw I/O performance of CoolVDS NVMe instances, you build a platform that is resilient, compliant, and incredibly fast.

Stop tolerating latency. Stop accepting manual configuration drift. Deploy a KVM instance in Oslo today and build a pipeline that actually works.