Keycloak Identity Provider

Keycloak provides enterprise-grade identity and access management with SAML 2.0 and OpenID Connect support.

Overview

Property Value

Hostname

keycloak-01.inside.domusdigitalis.dev

IP Address

10.50.1.80

Port

8443 (HTTPS)

OS

Fedora Cloud 43

Deployment

Docker Compose

VM Resources

Resource Value

vCPUs

2

Memory

4 GB

Disk

20 GB (qcow2)

Network

virbr0 (10.50.1.0/24)

Storage Pool

vms (/mnt/onboard-ssd)

Use Cases

  • SAML SSO for Cisco ISE admin portal

  • OIDC authentication for internal applications

  • Centralized identity management

  • MFA enforcement

VM Deployment

Prerequisites

  • Fedora Cloud image

  • KVM/libvirt hypervisor

  • virbr0 bridge network (10.50.1.0/24)

Step 1: Download Fedora Cloud Image

cd /mnt/onboard-ssd/vms

# List available versions
curl -sL https://download.fedoraproject.org/pub/fedora/linux/releases/43/Cloud/x86_64/images/ | grep -oE 'Fedora-Cloud[^"]+qcow2' | sort -u

# Download (use curl -LO to follow redirects)
sudo curl -LO https://download.fedoraproject.org/pub/fedora/linux/releases/43/Cloud/x86_64/images/Fedora-Cloud-Base-Generic-43-1.6.x86_64.qcow2

Step 2: Create Cloud-Init ISO

cd /mnt/onboard-ssd/isos

cat > meta-data <<EOF
instance-id: keycloak-01
local-hostname: keycloak-01
EOF

cat > user-data <<'EOF'
#cloud-config
hostname: keycloak-01
fqdn: keycloak-01.inside.domusdigitalis.dev
users:
  - name: evanusmodestus
    sudo: ALL=(ALL) NOPASSWD:ALL
    groups: wheel
    shell: /bin/bash
    lock_passwd: false
    plain_text_passwd: changeme
    ssh_authorized_keys:
      - ssh-ed25519 AAAAC3... your-key-here
chpasswd:
  expire: false
ssh_pwauth: true
EOF

genisoimage -output keycloak-01-cloud-init.iso -volid cidata -joliet -rock user-data meta-data
rm user-data meta-data

Step 3: Create VM

cd /mnt/onboard-ssd/vms

# Create disk from base image
sudo cp Fedora-Cloud-Base-Generic-43-1.6.x86_64.qcow2 keycloak-01.qcow2
sudo qemu-img resize keycloak-01.qcow2 20G

# Create VM
sudo virt-install \
  --name keycloak-01 \
  --vcpus 2 \
  --memory 4096 \
  --disk /mnt/onboard-ssd/vms/keycloak-01.qcow2,bus=virtio \
  --disk /mnt/onboard-ssd/isos/keycloak-01-cloud-init.iso,device=cdrom \
  --os-variant fedora-unknown \
  --network bridge=virbr0,model=virtio \
  --graphics none \
  --import \
  --noautoconsole

Step 4: Configure Static IP

Connect to console and configure networking:

sudo virsh console keycloak-01
# Login: evanusmodestus / changeme
# Change password immediately: passwd

# Create NetworkManager config
sudo tee /etc/NetworkManager/system-connections/enp1s0.nmconnection <<'EOF'
[connection]
id=enp1s0
type=ethernet
interface-name=enp1s0
autoconnect=true

[ipv4]
method=manual
addresses=10.50.1.80/24
gateway=10.50.1.1
dns=10.50.1.1

[ipv6]
method=disabled
EOF

sudo chmod 600 /etc/NetworkManager/system-connections/enp1s0.nmconnection
sudo nmcli con reload
sudo nmcli con up enp1s0

# Verify
nmcli
ping -c2 pfsense-01.inside.domusdigitalis.dev

Exit console: Ctrl+]

Step 5: Remove Cloud-Init CD

After initial setup, eject the cloud-init ISO:

sudo virsh change-media keycloak-01 sda --eject

Keycloak Installation

Directory Structure

/opt/keycloak/
├── docker-compose.yml
└── certs/
    ├── cert.pem          # Signed certificate
    └── key.pem           # Private key

Docker Compose

services:
  postgres:
    image: postgres:16
    container_name: keycloak-db
    environment:
      POSTGRES_DB: keycloak
      POSTGRES_USER: keycloak
      POSTGRES_PASSWORD: ${KC_DB_PASSWORD}
    volumes:
      - postgres_data:/var/lib/postgresql/data
    restart: unless-stopped

  keycloak:
    image: quay.io/keycloak/keycloak:latest
    container_name: keycloak
    command: start
    environment:
      KC_DB: postgres
      KC_DB_URL: jdbc:postgresql://postgres:5432/keycloak
      KC_DB_USERNAME: keycloak
      KC_DB_PASSWORD: ${KC_DB_PASSWORD}
      KC_HOSTNAME: keycloak-01.inside.domusdigitalis.dev
      KC_HTTPS_CERTIFICATE_FILE: /opt/keycloak/conf/cert.pem
      KC_HTTPS_CERTIFICATE_KEY_FILE: /opt/keycloak/conf/key.pem
      KEYCLOAK_ADMIN: admin
      KEYCLOAK_ADMIN_PASSWORD: ${KC_ADMIN_PASSWORD}
    ports:
      - "8443:8443"
    volumes:
      - ./certs:/opt/keycloak/conf:ro
    depends_on:
      - postgres
    restart: unless-stopped

volumes:
  postgres_data:

TLS Certificate (A-Z)

Modern browsers require certificates to have Subject Alternative Name (SAN) extension. Certificates with only CN will show "domain name does not match" errors.

Step 1: Generate CSR on keycloak-01

SSH into keycloak-01 and create the CSR with SAN:

ssh keycloak-01

# Create directory
sudo mkdir -p /opt/keycloak/certs
cd /opt/keycloak/certs

# Create OpenSSL config file with SAN (use sudo tee, not sudo cat >)
sudo tee keycloak.cnf << 'EOF'
[req]
default_bits = 2048
prompt = no
default_md = sha256
distinguished_name = dn
req_extensions = req_ext

[dn]
O = DomusDigitalis
OU = IdP
CN = keycloak-01.inside.domusdigitalis.dev

[req_ext]
subjectAltName = @alt_names

[alt_names]
DNS.1 = keycloak-01.inside.domusdigitalis.dev
DNS.2 = keycloak-01
EOF

# Generate private key and CSR using config
sudo openssl req -new -newkey rsa:2048 -nodes \
  -keyout key.pem \
  -out keycloak.csr \
  -config keycloak.cnf

# Verify SAN is in CSR
openssl req -in keycloak.csr -noout -text | grep -A2 "Subject Alternative Name"
# Expected: DNS:keycloak-01.inside.domusdigitalis.dev, DNS:keycloak-01
Use sudo tee for file creation with sudo, not sudo cat > (the redirect happens before sudo).

Step 2: Transfer CSR to Windows DC (Multi-Hop)

Since keycloak-01 may not have direct SSH access to home-dc01, use your workstation as a hop:

# From workstation - pull CSR from keycloak-01
scp keycloak-01:/opt/keycloak/certs/keycloak.csr /tmp/

# Push to Windows DC
scp /tmp/keycloak.csr home-dc01:C:/Certs/

Step 3: Sign with AD CS

SSH to Windows with -o ControlMaster=no to avoid stale socket issues. If you see mux_client_request_session: session request failed, remove the stale socket first:

rm ~/.ssh/sockets/Administrator@10.50.1.50-22
 
# Sign using WebServer template (preserves SAN from CSR)
ssh -o ControlMaster=no home-dc01 'certreq -submit -config "HOME-DC01.inside.domusdigitalis.dev\HOME-ROOT-CA" -attrib "CertificateTemplate:WebServer" "C:\Certs\keycloak.csr" "C:\Certs\keycloak.cer"'

# Expected output: Certificate retrieved(Issued) Issued

If Re-Submitting a CSR

Clean up old certificate files on the DC first to avoid overwrite prompts:

ssh -o ControlMaster=no home-dc01 'del C:\Certs\keycloak.* /Q'

To revoke a previously issued certificate:

# List issued certs for this CN
ssh -o ControlMaster=no home-dc01 'certutil -view -restrict "CommonName=keycloak-01.inside.domusdigitalis.dev" -out "SerialNumber,NotAfter"'

# Revoke old cert (4 = Superseded)
ssh -o ControlMaster=no home-dc01 'certutil -revoke <serial_number> 4'

Step 4: Retrieve Signed Certificate

# Copy back to workstation (multi-hop)
scp home-dc01:C:/Certs/keycloak.cer /tmp/

# Verify SAN is in signed certificate
openssl x509 -in /tmp/keycloak.cer -noout -text | grep -A2 "Subject Alternative Name"
# Expected: DNS:keycloak-01.inside.domusdigitalis.dev, DNS:keycloak-01

# Verify issuer and dates
openssl x509 -in /tmp/keycloak.cer -noout -subject -issuer -dates

Step 5: Deploy Certificate to keycloak-01

# Transfer to keycloak-01
scp /tmp/keycloak.cer keycloak-01:/tmp/

# Install certificate
ssh keycloak-01 'sudo mv /tmp/keycloak.cer /opt/keycloak/certs/cert.pem'

Step 6: Set Permissions and SELinux Context

Use chmod for permissions, not chown. The Keycloak container runs as UID 1000.

  • chown = change owner

  • chmod = change mode (permissions)

 
ssh keycloak-01 << 'EOF'
cd /opt/keycloak/certs

# Set ownership for Keycloak container (runs as UID 1000)
sudo chown 1000:1000 key.pem cert.pem

# Set permissions (chmod, not chown!)
sudo chmod 400 key.pem   # Read-only for owner
sudo chmod 444 cert.pem  # Read-only for all

# SELinux: Allow container access (Fedora)
sudo chcon -Rt svirt_sandbox_file_t /opt/keycloak/certs/

# Verify ownership, permissions, and SELinux context
ls -laZ /opt/keycloak/certs/
EOF

Expected output:

-r--r--r--. 1000 1000 ... cert.pem
-r--------. 1000 1000 ... key.pem
On Fedora with SELinux enforcing, volumes must have container_file_t or svirt_sandbox_file_t context for Docker access.

Secrets Management

Secrets are managed via dsec (age-encrypted) on the workstation. They are injected at deployment time and never persist on keycloak-01.

Add Secrets to dsec

# On workstation - add to dev/identity category
dsec edit d000 dev/identity

# Add:
# KC_DB_PASSWORD=<strong-password>
# KC_ADMIN_PASSWORD=<strong-password>

Deploy with Secrets

# Source secrets and deploy via SSH
eval "$(dsec source d000 dev/identity)"
ssh keycloak-01 "cd /opt/keycloak && sudo env KC_DB_PASSWORD='$KC_DB_PASSWORD' KC_ADMIN_PASSWORD='$KC_ADMIN_PASSWORD' docker compose up -d"
Using sudo env VAR=value preserves environment variables through sudo.

Verification

# Check containers
ssh keycloak-01 "sudo docker compose -f /opt/keycloak/docker-compose.yml ps"

# Check logs
ssh keycloak-01 "sudo docker logs keycloak 2>&1 | tail -20"

# Test HTTPS
curl -sk https://keycloak-01.inside.domusdigitalis.dev:8443/ -w "%{http_code}\n" -o /dev/null
# Expected: 302

# Verify certificate
echo | openssl s_client -connect keycloak-01.inside.domusdigitalis.dev:8443 2>/dev/null | openssl x509 -noout -subject -issuer

SSH Configuration

Add to dotfiles-optimus SSH config (base/ssh/.ssh/config):

Host keycloak-01
    HostName 10.50.1.80
    User evanusmodestus
    IdentityFile ~/.ssh/id_ed25519_sk_rk_d000
    IdentityFile ~/.ssh/id_ed25519_sk_rk_d000_secondary
    IdentityFile ~/.ssh/id_ed25519_d000
    PasswordAuthentication yes
    PreferredAuthentications publickey,password

Then restow: cd ~/dotfiles-optimus && stow -R base/ssh

ISE SAML Integration

Once Keycloak is running, configure ISE SAML SSO for admin portal authentication.

ISE uses Entity ID format CiscoISE/{UUID}, not the FQDN. Export ISE SP metadata first.

Quick Setup with netapi

# Load Keycloak secrets
eval "$(dsec source d000 dev/identity)"

# Download SAML metadata for ISE import
netapi keycloak get-saml-metadata domusdigitalis -o /tmp/keycloak-metadata.xml

# Manage Keycloak groups
netapi keycloak list-groups domusdigitalis
netapi keycloak create-group domusdigitalis ise-super-admin
netapi keycloak add-user-to-group domusdigitalis evanusmodestus ise-super-admin
netapi keycloak user-groups domusdigitalis evanusmodestus

Configuration Steps

  1. Export ISE SP metadata - Get the Entity ID from ISE

  2. Create Keycloak SAML client - Use Entity ID as Client ID

  3. Configure redirect URIs - Must include port 8443

  4. Import Keycloak metadata to ISE - Via GUI (SAML IdP is GUI-only)

  5. Map groups - Keycloak groups to ISE Admin Groups (case-sensitive!)

ISE Admin Group Mapping

Keycloak Group ISE Admin Group

ise-super-admin

Super Admin

ise-read-only

Read Only Admin

ise-helpdesk

Helpdesk Admin

ise-ers-admin

ERS Admin

Detailed Documentation

See PRJ-ISE-HOME-SAML project for complete ISE SAML configuration:

  • Keycloak SAML Client Configuration

  • ISE SAML IdP Configuration

  • Admin Group Mapping

  • Troubleshooting Guide

ISE SAML integration documentation is maintained in the PRJ-ISE-HOME-SAML project.

Troubleshooting

Browser Shows "Domain Name Does Not Match"

Cause: Certificate missing SAN (Subject Alternative Name). Modern browsers require SAN, not just CN.

Solution: Regenerate certificate with SAN. See Step 1: Generate CSR on keycloak-01.

# Verify SAN in current cert
echo | openssl s_client -connect keycloak-01.inside.domusdigitalis.dev:8443 2>/dev/null | openssl x509 -noout -text | grep -A2 "Subject Alternative Name"

SSH to Windows DC Hangs (ControlSocket Error)

Symptom:

mux_client_request_session: session request failed: Session open refused by peer
ControlSocket /home/user/.ssh/sockets/Administrator@10.50.1.50-22 already exists

Solution: Remove stale socket and use -o ControlMaster=no:

rm ~/.ssh/sockets/Administrator@10.50.1.50-22
ssh -o ControlMaster=no home-dc01 'certreq ...'

Keycloak Fails: AccessDeniedException key.pem

Symptom:

ERROR: Failed to load 'https-*' material: AccessDeniedException /opt/keycloak/conf/key.pem

Cause: Wrong ownership or permissions. Keycloak container runs as UID 1000.

Solution:

ssh keycloak-01 "sudo chown 1000:1000 /opt/keycloak/certs/*.pem && sudo chmod 400 /opt/keycloak/certs/key.pem && sudo chmod 444 /opt/keycloak/certs/cert.pem"

Permission Denied Creating Files with sudo

Symptom: sudo cat > file fails with permission denied.

Cause: Shell redirect (>) happens before sudo.

Solution: Use sudo tee:

sudo tee /path/to/file << 'EOF'
content here
EOF

certreq Prompts to Overwrite .rsp File

Cause: Previous certificate request files exist on the DC.

Solution: Delete old files first:

ssh -o ControlMaster=no home-dc01 'del C:\Certs\keycloak.* /Q'

KC_DB_PASSWORD Not Set Warning

Symptom:

WARN: The "KC_DB_PASSWORD" variable is not set. Defaulting to a blank string.

Solution: Start Keycloak with secrets from dsec:

eval "$(dsec source d000 dev/identity)"
ssh keycloak-01 "cd /opt/keycloak && sudo env KC_DB_PASSWORD='$KC_DB_PASSWORD' KC_ADMIN_PASSWORD='$KC_ADMIN_PASSWORD' docker compose up -d"

Browser HSTS Error After Certificate Change

Symptom: Firefox shows "HTTP Strict Transport Security (HSTS)" error, won’t allow exception.

Solution: Clear HSTS cache for the domain:

  1. Press Ctrl+Shift+H (History)

  2. Search for keycloak-01

  3. Right-click → Forget About This Site

  4. Restart browser

Verify Certificate Chain

# Full verification
echo | openssl s_client -connect keycloak-01.inside.domusdigitalis.dev:8443 2>/dev/null | openssl x509 -noout -subject -issuer -dates

# Expected:
# subject=O=DomusDigitalis, OU=IdP, CN=keycloak-01.inside.domusdigitalis.dev
# issuer=DC=dev, DC=domusdigitalis, DC=inside, CN=HOME-ROOT-CA

# Check SAN
echo | openssl s_client -connect keycloak-01.inside.domusdigitalis.dev:8443 2>/dev/null | openssl x509 -noout -text | grep -A2 "Subject Alternative Name"

Related Pages