Traefik Ingress Deployment

Deploy Traefik as the primary ingress controller for k3s services.

Overview

Traefik is a modern HTTP reverse proxy and load balancer that integrates natively with Kubernetes.

Feature Description

Auto-Discovery

Automatically discovers services via Kubernetes API

Let’s Encrypt

Built-in ACME for automatic TLS certificates

Middleware

Auth, rate limiting, headers, redirects

Dashboard

Real-time traffic visualization

Metrics

Prometheus metrics endpoint

Architecture

Traefik Ingress Flow

Prerequisites

# Verify k3s
kubectl get nodes

# Check if Traefik is already installed (k3s default)
kubectl get pods -n kube-system | grep traefik

# If Traefik exists, decide: use existing or replace
k3s includes Traefik by default. This runbook covers custom deployment with additional features.

Phase 1: Remove Default Traefik (Optional)

If you want full control over Traefik configuration:

1.1 Disable k3s Default Traefik

# On k3s node, edit k3s service
ssh k3s-master-01

sudo vi /etc/systemd/system/k3s.service
# Add to ExecStart: --disable=traefik

sudo systemctl daemon-reload
sudo systemctl restart k3s

# Verify removed
kubectl get pods -n kube-system | grep traefik

Phase 2: Namespace and Secrets

2.1 Create Traefik Namespace

kubectl create namespace traefik

2.2 Create Vault TLS Secret

kubectl create secret generic vault-tls \
  --namespace traefik \
  --from-file=ca.crt=/tmp/DOMUS-CA-CHAIN.pem

2.3 Create Default TLS Certificate

For internal services using Vault PKI:

# Issue wildcard cert from Vault
vault write -format=json pki_int/issue/domus-server \
  common_name="*.inside.domusdigitalis.dev" \
  ttl="8760h" > /tmp/wildcard-cert.json

# Extract cert and key
jq -r '.data.certificate' /tmp/wildcard-cert.json > /tmp/wildcard.crt
jq -r '.data.private_key' /tmp/wildcard-cert.json > /tmp/wildcard.key
jq -r '.data.ca_chain[]' /tmp/wildcard-cert.json >> /tmp/wildcard.crt

# Create k8s secret
kubectl create secret tls wildcard-tls \
  --namespace traefik \
  --cert=/tmp/wildcard.crt \
  --key=/tmp/wildcard.key

Phase 3: Helm Installation

3.1 Add Traefik Helm Repo

helm repo add traefik https://traefik.github.io/charts
helm repo update

3.2 Create Values File

traefik-values.yaml
# Deployment configuration
deployment:
  replicas: 1  # Single node, increase for HA
  kind: Deployment

# Pod configuration
podSecurityContext:
  fsGroup: 65532

# Resources
resources:
  requests:
    cpu: 100m
    memory: 128Mi
  limits:
    cpu: 500m
    memory: 512Mi

# Service configuration
service:
  enabled: true
  type: LoadBalancer
  spec:
    loadBalancerIP: 10.50.1.120  # k3s node IP

# Ports
ports:
  web:
    port: 80
    exposedPort: 80
    expose: true
    protocol: TCP
    redirectTo:
      port: websecure

  websecure:
    port: 443
    exposedPort: 443
    expose: true
    protocol: TCP
    tls:
      enabled: true

  metrics:
    port: 9100
    expose: false
    protocol: TCP

# TLS configuration
tlsStore:
  default:
    defaultCertificate:
      secretName: wildcard-tls

# Logs
logs:
  general:
    level: INFO
  access:
    enabled: true
    format: json

# Dashboard
ingressRoute:
  dashboard:
    enabled: true
    matchRule: Host(`traefik.inside.domusdigitalis.dev`)
    entryPoints:
      - websecure
    middlewares: []
    tls:
      secretName: wildcard-tls

# Prometheus metrics
metrics:
  prometheus:
    entryPoint: metrics
    addEntrypointsLabels: true
    addServicesLabels: true

# Providers
providers:
  kubernetesCRD:
    enabled: true
    allowCrossNamespace: true
  kubernetesIngress:
    enabled: true
    publishedService:
      enabled: true

# Additional arguments
additionalArguments:
  - "--api.insecure=false"
  - "--api.dashboard=true"
  - "--ping=true"
  - "--serversTransport.insecureSkipVerify=false"

# Security context
securityContext:
  capabilities:
    drop: [ALL]
  readOnlyRootFilesystem: true
  runAsGroup: 65532
  runAsNonRoot: true
  runAsUser: 65532

# Persistence for ACME (if using Let's Encrypt)
persistence:
  enabled: false  # Using Vault PKI instead

# RBAC
rbac:
  enabled: true

3.3 Install Traefik

helm install traefik traefik/traefik \
  --namespace traefik \
  --values traefik-values.yaml \
  --version 26.0.0

3.4 Verify Installation

# Check pods
kubectl get pods -n traefik

# Check services
kubectl get svc -n traefik

# Check IngressRoute for dashboard
kubectl get ingressroute -n traefik

Phase 4: DNS Configuration

4.1 Add DNS Entry

DNS records are added to BIND (authoritative DNS). See DNS Operations for full procedure.

ssh bind-01 "sudo nsupdate -l << 'EOF'
zone inside.domusdigitalis.dev
update add traefik.inside.domusdigitalis.dev. 3600 A 10.50.1.120
send
EOF"

4.2 Add Wildcard DNS (Optional)

For automatic service discovery, add wildcard record:

ssh bind-01 "sudo nsupdate -l << 'EOF'
zone inside.domusdigitalis.dev
update add *.k3s.inside.domusdigitalis.dev. 3600 A 10.50.1.120
send
EOF"
Wildcard DNS requires careful planning. Consider explicit records for each service instead.

Phase 5: Configure Ingress Routes

5.1 IngressRoute for Grafana

grafana-ingressroute.yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: grafana
  namespace: monitoring
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`grafana.inside.domusdigitalis.dev`)
      kind: Rule
      services:
        - name: prometheus-grafana
          port: 80
  tls:
    secretName: wildcard-tls

5.2 IngressRoute with Basic Auth

protected-ingressroute.yaml
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: basic-auth
  namespace: traefik
spec:
  basicAuth:
    secret: basic-auth-secret
---
apiVersion: v1
kind: Secret
metadata:
  name: basic-auth-secret
  namespace: traefik
type: kubernetes.io/basic-auth
stringData:
  username: admin
  password: changeme  # Use htpasswd format in production
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: protected-app
  namespace: default
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`app.inside.domusdigitalis.dev`)
      kind: Rule
      middlewares:
        - name: basic-auth
          namespace: traefik
      services:
        - name: my-app
          port: 80
  tls:
    secretName: wildcard-tls

5.3 IngressRoute with Rate Limiting

ratelimit-middleware.yaml
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: rate-limit
  namespace: traefik
spec:
  rateLimit:
    average: 100
    burst: 50
    period: 1m
    sourceCriterion:
      ipStrategy:
        depth: 1

Phase 6: Monitoring Integration

6.1 ServiceMonitor for Prometheus

traefik-servicemonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: traefik
  namespace: monitoring
  labels:
    release: prometheus
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: traefik
  namespaceSelector:
    matchNames:
      - traefik
  endpoints:
    - port: metrics
      interval: 30s
      path: /metrics

6.2 Grafana Dashboard

Import dashboard ID: 17346 (Traefik Official)

Troubleshooting

503 Service Unavailable

# Check if backend service exists
kubectl get svc -A | grep <service-name>

# Check endpoints
kubectl get endpoints -n <namespace> <service-name>

# Check Traefik logs
kubectl logs -n traefik -l app.kubernetes.io/name=traefik

Certificate Errors

# Verify TLS secret exists
kubectl get secret -n traefik wildcard-tls

# Check certificate details
kubectl get secret -n traefik wildcard-tls -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -text -noout | head -20

# Check Traefik sees the cert
kubectl logs -n traefik -l app.kubernetes.io/name=traefik | grep -i tls

Dashboard Not Accessible

# Check IngressRoute
kubectl get ingressroute -n traefik

# Port-forward directly
kubectl port-forward -n traefik svc/traefik 9000:9000

# Check if dashboard is enabled
kubectl get deployment -n traefik traefik -o yaml | grep -A5 dashboard

Resource Usage

| Component | Memory | CPU | |-----------|--------|-----| | Traefik | 128Mi-512Mi | 100m-500m |

Traefik is lightweight - single pod handles significant traffic.