Lab: Certificate Lifecycle — VMCA Subordinate CA & Rotation

Pre-flight: Audit current certificate state. SSH to vCenter → /usr/lib/vmware-vmafd/bin/vecs-cli store list → lists all certificate stores. For each store: vecs-cli entry list --store <name> → shows certificates with expiry dates. Alternatively: vCenter UI → Administration → Certificate Management → overview. Document current state: VMCA root cert expiry, machine SSL cert expiry, solution user cert expiry.

Generate VMCA subordinate CSR. SSH to vCenter → /usr/lib/vmware-vmca/bin/certificate-manager → Option 2: 'Replace VMCA Root Certificate with Custom Signing Certificate'. Follow prompts to generate a CSR. The CSR includes VMCA's public key and subject information. Save the CSR file — you'll submit it to the enterprise CA.

Sign CSR with enterprise CA. Submit the CSR to your enterprise CA: (a) For AD CS: Web enrollment → Advanced Certificate Request → paste CSR → select 'Subordinate Certification Authority' template → Submit. (b) For OpenSSL lab CA: openssl x509 -req -in vmca.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out vmca-signed.crt -days 3650 -extfile vmca-ext.cnf (with basicConstraints=CA:TRUE, keyUsage=keyCertSign,cRLSign). Download the signed certificate and CA chain.

Import signed certificate into VMCA. Return to certificate-manager → provide the signed VMCA certificate and the CA chain (root CA + any intermediate CAs). Certificate-manager: (a) replaces the VMCA root with the signed subordinate certificate, (b) automatically re-issues all leaf certificates (Machine SSL, solution users) signed by the new VMCA. This process takes 5-10 minutes. vCenter services restart automatically.

Verify certificate chain. After vCenter services restart: (a) browse to vCenter UI — browser should show trusted certificate (green lock) if enterprise CA is in the browser trust store; (b) SSH to vCenter: openssl s_client -connect localhost:443 -showcerts → verify chain: leaf cert → VMCA subordinate → enterprise root CA; (c) Check ESXi hosts: vCenter → Host → Configure → Certificate → should show 'Issued by: VMCA' with new dates.

Rotate ESXi host certificates. vCenter → Host → Configure → Certificate → Renew. vCenter contacts VMCA, generates a new leaf certificate for the host, and installs it. The host remains operational — no reboot required. Repeat for all hosts. For bulk rotation: use the vSphere API (POST /rest/vcenter/certificate-management/vcenter/tls) or VCF LCM certificate update workflow.

Configure certificate expiry monitoring. VCF Operations → alert definition → create symptom: 'Certificate Days to Expiry < 60'. Alert definition: 'Certificate-Expiry-Warning' → Criticality=Warning. Notification: email to security-team@lab.local. Create a second alert at 30 days with Criticality=Critical. This proactive monitoring prevents certificate-related outages (expired certificates cause service communication failures).

Test certificate trust. From a client machine: (a) curl -v https://<vcenter-fqdn> — verify TLS handshake succeeds without certificate warnings; (b) PowerCLI: Connect-VIServer <vcenter> — should connect without -Force parameter (trusted cert); (c) REST API: curl https://<vcenter>/rest/vcenter/vm — no certificate error. If trust fails: import the enterprise root CA certificate into the client trust store.

Document certificate lifecycle runbook. Record: (a) VMCA subordinate CA configuration procedure with screenshots; (b) certificate rotation schedule (annual recommended); (c) emergency rotation procedure (for compromised certificate); (d) certificate chain: list all CAs in the trust chain with expiry dates; (e) monitoring: alert thresholds and notification channels. This documentation is a VCAP exam deliverable and a production operations requirement.

Rollback test. In a lab environment, test rollback: revert VMCA to self-signed root using certificate-manager Option 4 (Reset all certificates). Observe: (a) all leaf certificates are re-issued with self-signed VMCA root; (b) browser shows untrusted certificate again; (c) existing API integrations may break (they trusted the enterprise chain). This validates your rollback procedure without affecting production.