CR-2026-02-25: Wazuh Credential Rotation
Change Summary
CR ID |
CR-2026-02-25-002 |
Date |
2026-02-25 |
Priority |
P0 - Emergency |
Type |
Security - Credential Rotation |
Status |
Completed |
Objective
Rotate Wazuh indexer credentials that were inadvertently exposed in Claude Code settings file.
Credential Architecture
Wazuh uses three separate credentials:
| Secret | Username | Component | Purpose |
|---|---|---|---|
|
admin |
OpenSearch (Indexer) |
Admin API access, cluster management |
|
kibanaserver |
Dashboard → Indexer |
Backend auth for dashboard queries |
|
wazuh-wui |
Manager API |
UI authentication |
Only indexer-cred was exposed.
Implementation Summary
Phase 0: Pre-Rotation Validation
# Store current password
CURRENT_PASS=$(ssh k3s-master-01 "kubectl -n wazuh get secret indexer-cred -o jsonpath='{.data.password}' | base64 -d")
# Test current access
ssh k3s-master-01 "kubectl -n wazuh exec wazuh-indexer-0 -c wazuh-indexer -- \
curl -sk -u admin:$CURRENT_PASS https://localhost:9200/_cluster/health" | jq -r '.status'
Phase 1: Generate New Credential
NEW_PASS=$(tr -dc 'A-Za-z0-9' </dev/urandom | head -c 32)
export NEW_PASS
Phase 2: Store in gopass (Source of Truth)
cat << EOF | gopass insert -m v3/domains/d000/k3s/wazuh/indexer
$NEW_PASS
---
username: admin
service: wazuh-indexer (OpenSearch)
k8s_secret: indexer-cred
k8s_namespace: wazuh
rotated: 2026-02-25
EOF
Phase 3: Update Kubernetes Secret
NEW_PASS_B64=$(echo -n "$NEW_PASS" | base64)
ssh k3s-master-01 "kubectl -n wazuh patch secret indexer-cred -p '{\"data\":{\"password\":\"$NEW_PASS_B64\"}}'"
Phase 4: Update OpenSearch Internal User Database
|
OpenSearch stores password hashes internally. Updating K8s secret alone does NOT change the password. Must run |
# Generate bcrypt hash locally (container's hash.sh lacks Java)
BCRYPT_HASH=$(python3 -c "import bcrypt; print(bcrypt.hashpw(b'$NEW_PASS', bcrypt.gensalt(rounds=12)).decode())")
# Update internal_users.yml via cat (sed -i fails on overlayfs)
ssh k3s-master-01 "kubectl -n wazuh exec wazuh-indexer-0 -c wazuh-indexer -- \
cat /usr/share/wazuh-indexer/config/opensearch-security/internal_users.yml" > /tmp/internal_users.yml
# Edit with awk (handles $ in bcrypt hash correctly)
awk -v hash="$BCRYPT_HASH" '
/^admin:/ {found=1}
found && /hash:/ {sub(/hash:.*/, "hash: \"" hash "\""); found=0}
{print}
' /tmp/internal_users.yml >| /tmp/internal_users.yml.new && mv /tmp/internal_users.yml.new /tmp/internal_users.yml
# Write to container /tmp (config dir is read-only)
ssh k3s-master-01 "kubectl -n wazuh exec -i wazuh-indexer-0 -c wazuh-indexer -- \
sh -c 'cat > /tmp/internal_users.yml'" < /tmp/internal_users.yml
# Apply with securityadmin.sh
ssh k3s-master-01 "kubectl -n wazuh exec wazuh-indexer-0 -c wazuh-indexer -- \
env JAVA_HOME=/usr/share/wazuh-indexer/jdk \
/usr/share/wazuh-indexer/plugins/opensearch-security/tools/securityadmin.sh \
-f /tmp/internal_users.yml -t internalusers -icl -nhnv \
-cacert /usr/share/wazuh-indexer/config/certs/root-ca.pem \
-cert /usr/share/wazuh-indexer/config/certs/admin.pem \
-key /usr/share/wazuh-indexer/config/certs/admin-key.pem"
Phase 5: Restart Pods (Dependency Order)
# 1. Indexer first
ssh k3s-master-01 "kubectl -n wazuh rollout restart statefulset wazuh-indexer && \
kubectl -n wazuh rollout status statefulset wazuh-indexer --timeout=180s"
# 2. Wait for API ready
for i in {1..30}; do
ssh k3s-master-01 "kubectl -n wazuh exec wazuh-indexer-0 -c wazuh-indexer -- \
curl -sk -u admin:$NEW_PASS https://localhost:9200/_cluster/health" 2>/dev/null | \
jq -e '.status' &>/dev/null && break
sleep 5
done
# 3. Manager master
ssh k3s-master-01 "kubectl -n wazuh rollout restart statefulset wazuh-manager-master"
# 4. Manager worker
ssh k3s-master-01 "kubectl -n wazuh rollout restart statefulset wazuh-manager-worker"
# 5. Dashboard last
ssh k3s-master-01 "kubectl -n wazuh rollout restart deployment wazuh-dashboard"
Phase 6: Post-Rotation Validation
# Verify NEW password works
ssh k3s-master-01 "kubectl -n wazuh exec wazuh-indexer-0 -c wazuh-indexer -- \
curl -sk -u admin:$NEW_PASS https://localhost:9200/_cluster/health" | jq -r '.status'
# Expected: green
# Verify OLD password rejected
ssh k3s-master-01 "kubectl -n wazuh exec wazuh-indexer-0 -c wazuh-indexer -- \
curl -sk -u admin:$CURRENT_PASS https://localhost:9200/_cluster/health" 2>&1 | \
grep -qE "Unauthorized|401" && echo "✓ Old password rejected"
CLI Mastery: Container Workarounds
| Problem | Solution |
|---|---|
|
Use |
|
Use |
Container’s hash.sh fails (no Java) |
Generate bcrypt locally: |
ConfigMap/Secret mounts are read-only |
Write to |
Bcrypt hash contains |
Use |
|
Set |
Key Lessons
| Issue | Mitigation |
|---|---|
K8s secret ≠ OpenSearch password |
Must run |
Hardcoded secrets in config files |
Use environment variables: |
Flat gopass structure |
Use resource-based paths: |
No pre/post validation |
Always test access BEFORE and AFTER rotation |