Academy/vSphere Foundation 9.0 Support (2V0-18.25)/Lab: VCSA Troubleshooting & Certificate Renewal
This lab targets VCF 9.0

Lab: VCSA Troubleshooting & Certificate Renewal

VCF 9.0Intermediatevcp-foundation⏱ 105 min

VVF 9.0 support lab (2V0-18.25). Built for a Holodeck VVF instance.

Objectives

  • Check vCenter service health with service-control
  • Read the Machine SSL certificate expiry from VECS and renew it after a file-based backup
  • Check /storage free space and the vmware-vpostgres service

Prerequisites

Holodeck VVF 9.0 instance; SSH to the vCenter appliance enabled; a current file-based backup of vCenter

Lab Environment

Holodeck vSphere Foundation 9.0 instance (New-HoloDeckInstance -VVF; -vSANMode OSA by default, ESA where the physical layer supports it): vCenter managing one vSAN cluster of nested ESX hosts, plus VCF Operations. VVF has no SDDC Manager and no NSX.

Credentials

SystemUsernamePassword
vCenteradministrator@vsphere.localHolodeck documented default (Post Deployment table)
VCF OperationsadminHolodeck documented default (Post Deployment table)
Nested ESX hostsrootHolodeck documented default (Post Deployment table)

Tasks

Task 1 vCenter health checks and Machine SSL renewal

VCSA troubleshooting requires understanding the Photon OS layer, PostgreSQL database, and vCenter services. Certificate issues are the most common VCSA problem — expired or mismatched certificates cause service failures across the entire VCF stack.
Step 1

SSH to VCSA: ssh root@vcsa_hostname

Step 2

Check services: service-control --status --all | grep -i "vpxd\|ui\|vpostgres"

Step 3

View the Machine SSL certificate expiry — it lives in VECS, not in a legacy file: /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store MACHINE_SSL_CERT --alias __MACHINE_CERT | openssl x509 -noout -enddate. Cross-check in the vSphere Client under Administration > Certificates > Certificate Management.

Step 4

Check disk space: df -h /storage (should show >20% free)

Step 5

Verify the database service: service-control --status vmware-vpostgres (should be running — the service name is vmware-vpostgres).

Step 6

Restart the vSphere Client service: service-control --restart vsphere-ui. The vSphere Client is unavailable for a few minutes; hosts and VMs are unaffected (hosts only show disconnected after a vpxd restart).

Step 7

After the restart, log in to the vSphere Client again. Then, after a file-based backup, renew the Machine SSL certificate from Administration > Certificates > Certificate Management (or with /usr/lib/vmware-vmca/bin/certificate-manager option 3 in the lab) and confirm the new 'Not After' date with the vecs-cli command from step 3.

Validation Gate

Check: Verify VCSA health: all services running, certificates not expiring within 30 days, disk partitions below 80%, and PostgreSQL database accessible

Expected: service-control --status shows all services green. Certificate Manager shows certificates valid for >30 days. df -h shows all partitions below 80%. psql can connect to VCDB.

Common Errors

Renewing certificates without checking the full certificate chain
Fix: Know what each store holds before renewing: the Machine SSL certificate (VECS store MACHINE_SSL_CERT) is what users and hosts see and can be replaced on its own through Certificate Management or certificate-manager; the STS signing certificate and the solution-user certificates are separate and are renewed by their own options. Renewing one does not force the others, but let a failed renewal roll back and check service-control --status --all afterwards.
Not backing up VCSA before certificate operations
Fix: Certificate renewal can fail mid-process, leaving vCenter in an inconsistent state. Always take a VCSA file-based backup (or VM snapshot) before any certificate operation. Recovery from a failed certificate renewal without backup requires manual certificate manipulation.
Ignoring VCSA service health check output
Fix: Run 'service-control --status' to check all vCenter services. Common issues: vpxd stopped (vCenter unavailable), vmware-stsd stopped (SSO broken), vmware-vpostgres stopped (database down). Address service failures in dependency order: database first, then SSO, then vpxd.
Not monitoring VCSA disk space
Fix: VCSA has multiple disk partitions. The /storage/log partition filling up is the most common cause of vCenter instability. Monitor with 'df -h' or vCenter alarm. When log partition exceeds 80%: rotate logs, identify chatty services, and consider increasing log partition size via VCSA management interface.

Final Validation

Gate: every check below passes with the evidence recorded in the lab log.

✓ service-control --status --all → All required services running

✓ Machine SSL certificate → New expiry date shown in VECS and in Administration > Certificates

✓ df -h /storage → Partitions below 80% used

✓ vmware-vpostgres → Running

Cleanup / Restore

• Disable SSH on the vCenter appliance again

Design Reflection (VCDX)

vCenter availability is critical — a VCSA failure impacts management of the entire domain. VCDX designs should include VCSA monitoring, certificate lifecycle management, and backup/recovery procedures.

Requirements

  • Monitor VCSA service health and disk space
  • Manage certificate lifecycle proactively
  • Maintain VCSA backup for recovery

Constraints

  • Certificate renewal must follow dependency order
  • Log partition can fill rapidly during service issues
  • VCSA backup must be current before certificate operations

Assumptions

  • VCSA backup runs daily via file-based backup
  • Certificate expiry monitoring is configured in VCF Operations

Risks

  • Certificate expiry cascade across VCF components
  • VCSA instability from log partition exhaustion

⚠ Known Pitfalls

Attempting certificate renewal without a backup — a failed renewal can leave vCenter unrecoverable without manual intervention.
Ignoring VCSA disk space alerts until the log partition is 100% full — at that point, vCenter services crash.

References

Was this page useful?
Type to search. ↑ ↓ to move, Enter to open, Esc to close.