Academy/VCF 9.0 Support (2V0-15.25)/Compute Core Concepts — ESXi Host Architecture
This lab targets VCF 9.0

Compute Core Concepts — ESXi Host Architecture

VCF 9.0Beginnersupportadmin⏱ 60 min

VCFFTS9 course content — VCF 9.0 focused

Objectives

  • Describe ESXi host architecture, security features, and boot bank layout
  • List hardware requirements for ESX 9.0 and vSAN
  • Perform ESXi host preparation for VCF (network, certificates, NTP)
  • Explain vCenter Appliance architecture and the vpxd→vpxa→hostd communication chain
  • Describe vCenter SSO and Enhanced Linked Mode (up to 15 vCenter instances)
  • Differentiate VSS from VDS and explain NIOC
  • Identify snapshot types by datastore (VMFSsparse, SEsparse, vsanSparse)
  • Explain vMotion including vGPU migration (up to 60 Gbps)
  • Describe vSphere Lifecycle Manager image-based management in 9.0

Prerequisites

Active Holodeck VCF 9.0 lab or access to VCF documentation

Required skills:

  • Basic VMware terminology

Tasks

Task 1 ESXi Host Architecture & Configuration

ESXi host preparation is where most VCF deployments fail. Certificate regeneration, NTP sync, and DNS resolution must be perfect before the VCF Installer can validate and commission hosts. A single misconfigured host blocks the entire bring-up.

Understand ESXi architecture, HW requirements, and VCF-specific host preparation

Step 1
ESXi Architecture

ESX is a bare-metal hypervisor. Security features: host-based firewall, memory hardening, kernel module integrity, TPM 2.0, UEFI secure boot, encrypted core dumps. Boot bank layout: /bootbank, /altbootbank (dual-bank for patching), /productLocker (VMware Tools ISOs), /var/core (core dumps). Small disk footprint with quick boot for faster patching.

Step 2
Hardware Requirements

ESX 9.0 minimum requirements: supported server platform (check VMware Compatibility Guide), 2+ CPU cores, 8GB RAM (12GB production recommended), 1+ GbE NIC, 32GB persistent boot disk. For vSAN: disks and controllers must be compatible per VMware Compatibility Guide. Memory Tiering: PCIe NVMe devices can serve as second memory layer for more in-memory computing.

[HOLODECK NOTE] In Holodeck, ESXi runs as nested VMs on a physical host. Hardware compatibility checks (VMware Compatibility Guide) do not apply — nested ESXi uses virtual hardware. CPU features are passed through from the physical host. Memory is allocated from the physical host's RAM pool — Holodeck's documented single-site minimums are far higher than a nested-ESXi rule of thumb suggests: 325 GB RAM / 32 cores for VCF 9.0 (with VCF Automation on vSAN ESA), 256 GB for VVF 9.0, 650 GB for VCF 9.1 — see the Holodeck prerequisites page.

Step 3
ESXi Host Preparation for VCF

For management domain: interactive ESX installation on all hosts. Post-install config via DCUI and Host Client: (1) Management network — adapter selection, VLAN ID, IPv4/IPv6, hostname, DNS; (2) VM Network port group — set VLAN ID on the vSphere Standard Switch so the VCF Installer can reach the hosts during the automated deployment; (3) Certificate regeneration — in the Host Client enable SSH, run /sbin/generate-certificates over SSH, reboot the host, then disable SSH again (VCF 9.0.1 docs p.1269-1270); (4) NTP configuration (UDP 123) or PTP (UDP 319/320 for microsecond accuracy). Ensure no Custom DNS Suffixes defined.

Step 4
Remote Access & Security

ESX firewall activated by default — blocks all except enabled services. SSH and Shell managed by admin users. Lockdown mode: host accessible only via DCUI or through vCenter (no direct remote login). Services management: START/STOP/RESTART, startup policy (manual, start with host, start/stop with port usage).

Validation Gate

Check: Verify ESXi host is VCF-ready: FQDN matches DNS (forward + reverse), certificates regenerated, NTP synchronized, no Custom DNS Suffix

Expected: All four checks pass. Host is ready for VCF Installer validation and commissioning.

Common Errors

Forgetting to regenerate ESXi certificates after hostname change
Fix: After setting hostname and DNS: Host Client > Actions > Services > Enable Secure Shell, SSH in and run /sbin/generate-certificates, reboot the host, then Disable Secure Shell (the documented procedure — VCF 9.0.1 docs p.1269-1270; service restarts are not the documented step). Without this, the VCF Installer's host certificate validation fails (it checks the certificate against the FQDN entered in the wizard) with 'certificate hostname mismatch'.
Leaving Custom DNS Suffix configured on ESXi host
Fix: A Custom DNS Suffix in ESXi DCUI causes FQDN resolution mismatches. The VCF Installer expects the host FQDN to match forward and reverse DNS exactly. Remove any Custom DNS Suffix before commissioning.
NTP not synchronized before bring-up
Fix: VCF bring-up requires all hosts within 30 seconds of NTP source. Use 'esxcli system time get' and verify NTP service is running. Time drift causes certificate validation failures and vSAN cluster formation issues.
Using DHCP instead of static IP for ESXi management
Fix: VCF requires static IP assignment for all ESXi management interfaces. DHCP-assigned addresses cause host commissioning failures because SDDC Manager cannot reliably reach hosts if IPs change.

Task 2 vCenter Architecture & Management

Understanding the vpxd → vpxa → hostd communication chain is essential for troubleshooting connectivity issues between vCenter and ESXi hosts. When a host shows 'disconnected' in vCenter, the root cause is always in this chain.

Understand vCenter Appliance architecture, services, and SSO

Step 1
vCenter Appliance Architecture

vCenter Appliance = prepackaged Linux VM: Photon OS + PostgreSQL database + vCenter services. All services on single VM. Services include: vCenter, vSphere Client, Authentication Services, License service, Content Library, vSphere Lifecycle Manager. Deploy on existing ESX host, select appliance size for environment + storage size for DB.

Step 2
ESXi-vCenter Communication

Communication chain: vSphere Client → vpxd (vCenter) → vpxa (host agent, auto-started when host added) → hostd (host daemon). hostd manages all local operations: VM creation, power state, storage visibility. Direct host access (bypassing vCenter) communicates with hostd directly via VMware Host Client.

Step 3
vCenter SSO & Enhanced Linked Mode

SSO provides centralized authentication: identity sources (AD, LDAP, local OS), security tokens, site/domain concepts. Enhanced Linked Mode: up to 15 vCenter instances in single SSO domain for unified inventory, role management, and search across data centers. IMPORTANT: Enhanced Linked Mode (ELM) is DEPRECATED in vCenter 9.0 and will be removed in a future release. ELM is still supported in VCF 9.0 only to allow smooth upgrades of existing deployments. New deployments should use VCF Operations Fleet Management for multi-instance management instead of ELM.

Validation Gate

Check: Trace a VM power-on request from vSphere Client through vpxd, vpxa, and hostd

Expected: vSphere Client → vpxd (vCenter service, validates permissions, updates DB) → vpxa (host agent on target ESXi, receives task) → hostd (executes power-on, updates VM state) → status propagates back up the chain

Common Errors

Confusing Enhanced Linked Mode deprecation with immediate removal
Fix: ELM is deprecated in vCenter 9.0, not removed. Existing ELM deployments continue to function. However, new VCF 9.0 deployments should NOT configure ELM — use VCF Operations Fleet Management instead for multi-instance visibility.
Not understanding SSO domain scope for RBAC
Fix: SSO domain defines the RBAC boundary. Permissions assigned in one vCenter within an SSO domain apply across all linked vCenters. In the VCF 9.0 design, each vCenter (management and every workload domain) has a distinct vCenter Single Sign-On domain; vCenter instances can be linked for tag sharing through vCenter Linking groups in VCF Operations. VCF Instances upgraded from an earlier version may still have shared SSO domains (ELM) — know which model you are troubleshooting before reasoning about RBAC scope.

Task 3 vSphere Networking, VMs & Clusters

Snapshot management is a common operational pain point. Understanding snapshot types per datastore is critical for troubleshooting performance issues caused by snapshot chains and for backup architecture design.

Cover networking constructs, VM operations, and cluster capabilities

Step 1
vSphere Networking

VSS (Standard Switch): per-host, manual config. VDS (Distributed Switch): centrally managed across hosts, supports port groups spanning hosts, NetFlow, port mirroring, LACP. VMkernel adapters for: management, vMotion, vSAN, fault tolerance logging, provisioning traffic. NIC teaming: load-based, source port ID, source MAC hash, explicit failover. Network I/O Control (NIOC): bandwidth shares and reservations per traffic type.

[HOLODECK NOTE] Holodeck uses virtual switches on the physical host to provide connectivity to nested ESXi. VDS in nested ESXi connects to virtual port groups on the physical host's vSwitch. NIC teaming policies in nested ESXi operate on virtual NICs — failover and load balancing behavior differs from physical multi-NIC configurations. NIOC bandwidth reservations have no real effect in nested environments.

Step 2
Virtual Machine Operations

VM files: .vmx (config), .vmdk (disk descriptor), -flat.vmdk (data), .nvram (BIOS/EFI), .vmsd (snapshot list), .vmsn (snapshot state), .vmem (memory state). Snapshot types by datastore: VMFSsparse (VMFS5 <2TB, 512-byte blocks), SEsparse (VMFS6, 4KB blocks, space-efficient with unmap), vsanSparse (vSAN OSA redo-log format); vSAN ESA uses native snapshots (B-tree structured, no redo-log chain, deletes without data movement). CBT (Changed Block Tracking): VMkernel feature for incremental backups — tracks changed blocks, reduces restore time.

Step 3
Cluster Operations & Live Migration

Cluster: two host limits apply, each with its own scope. (1) vSphere cluster (ESX 9.0): 96 hosts per cluster — VMware Configuration Maximums tool, VMware Cloud Foundation > ESX 9.0.0, Cluster and Resource Pool Maximums (https://configmax.broadcom.com/guest?vmwareproduct=ESX&release=9.0.0&categories=2-0), whose note says the 96-host limit applies from 7.0U1 onwards and is valid only for non-vSAN clusters (7.0 GA was 64). The VCF 9.0.1 docs refer ESX host maximums to VMware Configuration Maximums (p.1492), and the Broadcom VCF blog 'Upgrading VMware Cloud Foundation 5.2 to 9.0: The Top 10 Questions Answered' (18 Dec 2025, https://blogs.vmware.com/cloud-foundation/2025/12/18/upgrading-vmware-cloud-foundation-5-2-to-9-0-the-top-10-questions-answered/) answers the consolidated-design question (Q6) for VCF 9.0: no documented limits beyond those of vSphere itself — 96 hosts per cluster and 2500 hosts per vCenter. (2) vSAN cluster: 64 hosts — vSAN scales 'to clusters with up to 64 nodes' (VCF 9.0.1 docs p.454) and 'up to 64 hosts per vSAN cluster' (p.5274). Design vSAN clusters to 64; only a non-vSAN cluster (external storage) can go to 96. The 'Max Hosts per Cluster 64' row on p.970 belongs to the VCF Edge design-options table (1-node, 2-node with external storage, 3-node vSAN, 4-node fleet), so it scopes VCF Edge deployments, not every VCF deployment type. DRS: automated load balancing. HA: host monitoring, admission control, VM restart on failure. vMotion: live migration with zero downtime. vGPU vMotion: multi-parallel TCP connections up to 60 Gbps (6x faster), zero-copy technique, precopy for minimal downtime. EVC (Enhanced vMotion Compatibility): CPU feature masking for heterogeneous clusters. Migration types: compute only, storage only, both, cross-vCenter export.

[HOLODECK NOTE] In Holodeck's nested environment: vMotion works but bandwidth is limited by virtual NIC throughput, not physical 25GbE. The 60 Gbps vGPU vMotion figure applies to bare-metal with dedicated NICs — Holodeck nested NICs will not approach this. Cluster limits (96 hosts non-vSAN, 64 hosts vSAN) are theoretical — Holodeck typically runs 3-4 nested hosts due to RAM constraints.

Step 4
Content Libraries & vSphere Lifecycle Manager

Content libraries: local (admin-controlled), published (available for subscription), subscribed (sync from publisher). Templates stored as OVF or VM templates. vSphere 9.0: library migration to new datastore supported. vSphere Lifecycle Manager: image-based management (no more baselines in 9.0), desired state model — cluster image includes base ESXi + vendor add-on + components + firmware.

Validation Gate

Check: Name the snapshot type for each datastore: VMFS5, VMFS6, vSAN ESA

Expected: VMFS5 (<2TB): VMFSsparse (512-byte blocks). VMFS6: SEsparse (4KB blocks, space-efficient with UNMAP). vSAN OSA: vsanSparse (redo-log). vSAN ESA: native snapshots (B-tree, no redo-log chain; deletes involve no data movement).

Common Errors

Running VMs on snapshots for extended periods in production
Fix: Snapshots are for short-term use (backups, patching windows). A snapshot chain growing beyond 3 levels causes exponential I/O overhead — each read must traverse the chain. Monitor snapshot age via VCF Operations alerts. vSAN ESA native snapshots avoid chain traversal and delete without data movement, so long-running consolidation is not expected there; the chain penalty applies to VMFS and vSAN OSA formats.
Confusing vMotion bandwidth in nested vs bare-metal
Fix: The 60 Gbps vGPU vMotion figure applies to bare-metal with dedicated 25GbE NICs and RDMA. In Holodeck nested environments, vMotion bandwidth is limited by virtual NIC throughput (typically 1-10 Gbps). Do not cite nested lab migration times as production performance.
Not understanding image-based lifecycle management in vSphere 9.0
Fix: vSphere 9.0 removes baseline-based patching entirely. All host updates are image-based via vSphere Lifecycle Manager (vLCM). The cluster image includes: base ESXi + vendor add-on (firmware/drivers) + components. This is a significant operational change from legacy Update Manager baselines.

Design Reflection (VCDX)

Compute architecture forms the foundation of every VCF design. VCDX panelists test whether you understand ESXi host sizing, cluster limits, and lifecycle management. The shift to image-based management in vSphere 9.0 is a frequent discussion point.

Requirements

  • Understand ESXi architecture, security features, and VCF preparation
  • Master vCenter communication chain for troubleshooting
  • Know cluster limits, migration types, and snapshot behavior

Constraints

  • ESX 9.0 requires image-based lifecycle management (no baselines)
  • ELM deprecated — use Fleet Management for multi-instance
  • Cluster limits: 96 hosts per vSphere cluster, non-vSAN only (Configuration Maximums tool, ESX 9.0.0; applied to VCF 9.0 consolidated designs in the Broadcom VCF blog of 18 Dec 2025 (Q6)); 64 per vSAN cluster (p.454, p.5274); 64 in the VCF Edge design options (p.970)

Assumptions

  • Hardware meets VMware Compatibility Guide requirements
  • NTP, DNS, and certificates are configured correctly before bring-up

Risks

  • Snapshot chain growth causing I/O performance degradation
  • Certificate mismatch blocking host commissioning
  • Confusing nested lab performance with production benchmarks

⚠ Known Pitfalls

Quoting nested lab performance numbers (vMotion speed, I/O throughput) as production expectations — always distinguish Holodeck measurements from bare-metal baselines.
Forgetting that vSphere 9.0 removes baseline-based patching — all host lifecycle is image-based via vLCM.
Overlooking ESXi certificate regeneration as a pre-commissioning step — this is a leading cause of VCF Installer host-validation failures in the field.

References

Was this page useful?
Type to search. ↑ ↓ to move, Enter to open, Esc to close.