MetalLB Load Balancer for k3s

Overview

MetalLB provides LoadBalancer service type support for bare metal Kubernetes clusters.

Why MetalLB?

  • k3s on bare metal has no cloud LoadBalancer

  • Traefik LoadBalancer service shows node IP but doesn’t bind to 80/443

  • VXLAN tunnel mode prevents direct BPF attachment to host interface

  • MetalLB uses ARP (L2 mode) or BGP to advertise external IPs

Prerequisites

  • k3s cluster running with Cilium CNI

  • Helm 3 installed

  • IP range available in your network (not used by DHCP)

Phase 1: Install MetalLB

1.1 Add Helm Repository

helm repo add metallb https://metallb.github.io/metallb
helm repo update

1.2 Create Namespace and Install

helm install metallb metallb/metallb -n metallb-system --create-namespace

1.3 Wait for Pods

kubectl -n metallb-system get pods -w
Expected output (wait for Running)
NAME                                  READY   STATUS    RESTARTS   AGE
metallb-controller-xxx                1/1     Running   0          30s
metallb-speaker-xxx                   1/1     Running   0          30s

Press Ctrl+C when both are Running.

Phase 2: Configure IP Address Pool

2.1 Choose IP Range

Select IPs from your management network that are NOT in DHCP scope.

Network DHCP Range MetalLB Range (example)

10.50.1.0/24

10.50.1.100-10.50.1.119

10.50.1.130-10.50.1.140
Verify these IPs are not in use: for i in {130..140}; do ping -c1 -W1 10.50.1.$i && echo "IN USE: 10.50.1.$i"; done

2.2 Create IPAddressPool and L2Advertisement

cat <<'EOF' | kubectl apply -f -
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: mgmt-pool
  namespace: metallb-system
spec:
  addresses:
  - 10.50.1.130-10.50.1.140
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: mgmt-l2
  namespace: metallb-system
spec:
  ipAddressPools:
  - mgmt-pool
EOF

2.3 Verify Configuration

kubectl -n metallb-system get ipaddresspool
kubectl -n metallb-system get l2advertisement

Phase 3: Verify LoadBalancer Gets External IP

3.1 Check Traefik Service

kubectl -n kube-system get svc traefik
Before MetalLB
NAME      TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)
traefik   LoadBalancer   10.43.95.102   10.50.1.120   80:32327/TCP,443:32503/TCP

The EXTERNAL-IP was the node IP. After MetalLB, it should get a pool IP.

3.2 Restart Traefik to Pick Up New IP

kubectl -n kube-system rollout restart deployment traefik
kubectl -n kube-system get svc traefik -w
After MetalLB (expected)
NAME      TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)
traefik   LoadBalancer   10.43.95.102   10.50.1.130    80:32327/TCP,443:32503/TCP

The EXTERNAL-IP should now be from the MetalLB pool (e.g., 10.50.1.130).

Phase 4: Update DNS

If Traefik gets a new IP from MetalLB pool, update BIND DNS records:

ssh bind-01 "sudo nsupdate -l << 'EOF'
zone inside.domusdigitalis.dev
update add grafana.inside.domusdigitalis.dev. 3600 A 10.50.1.130
update add prometheus.inside.domusdigitalis.dev. 3600 A 10.50.1.130
update add alertmanager.inside.domusdigitalis.dev. 3600 A 10.50.1.130
send
EOF"

Or update existing records if IPs changed.

Phase 5: Test Standard Ports

# Test port 443 (should work now!)
curl -kI https://grafana.inside.domusdigitalis.dev

# Test port 80 redirect
curl -I http://grafana.inside.domusdigitalis.dev

Phase 6: Firewall Rules (MetalLB IP)

If using a NEW IP from the pool, add firewall rules on that IP or ensure VyOS allows traffic to the MetalLB range.

# On k3s node (if MetalLB IP is different from node IP)
# Usually not needed - MetalLB uses ARP to claim the IP

# Verify ARP
ssh k3s-master-01 "ip neigh | grep 10.50.1.130"

Troubleshooting

No External IP Assigned

# Check MetalLB controller logs
kubectl -n metallb-system logs -l app.kubernetes.io/component=controller

# Check speaker logs (ARP announcements)
kubectl -n metallb-system logs -l app.kubernetes.io/component=speaker

IP Conflict

If another device has the IP:

# Check ARP table
arp -a | grep 10.50.1.130

# Ping to see if responsive
ping 10.50.1.130

Service Still Shows Node IP

# Delete and recreate service
kubectl -n kube-system delete svc traefik
# Traefik deployment will recreate it, or:
kubectl -n kube-system rollout restart deployment traefik

Architecture

MetalLB L2 Architecture
Figure 1. MetalLB L2 Architecture

Key Concepts

Concept Description

L2 Mode

MetalLB uses ARP (IPv4) or NDP (IPv6) to claim IPs. Simple, no BGP needed.

IPAddressPool

Range of IPs MetalLB can assign to LoadBalancer services.

L2Advertisement

Tells MetalLB which pools to advertise via ARP/NDP.

Speaker

DaemonSet that responds to ARP requests for VIPs.

Controller

Assigns IPs from pools to services.

References