Deploy VCF 9.0.x Management Domain Using Holodeck Toolkit
Objectives
- Configure the Holodeck Toolkit deployment parameters for a VCF 9.0.x management domain
- Execute a successful Holodeck-driven nested VCF deployment from Prepare through Start
- Validate that all management domain components (vCenter, SDDC Manager, NSX Manager, ESXi hosts) are healthy post-deployment
- Interpret common deployment failures using Holodeck logs and the community knowledge base
- Articulate the nested architecture's relationship to production VCF design constraints
Prerequisites
Physical ESXi 8.0.x host with Holodeck Toolkit installed and content pack staged (holodeck-01 completed). No existing Holodeck instance running. Sufficient resources: minimum 512 GB RAM, 64 logical cores, 2 TB SSD/NVMe datastore. Internet connectivity or offline depot configured.
Prior labs: holodeck-01
Required skills:
- ESXi host management via vSphere Client
- Basic PowerShell navigation and command execution
- Understanding of VCF management domain architecture (Cloud Builder, SDDC Manager, vCenter, NSX)
- Familiarity with nested virtualization concepts
Lab Environment
Single physical ESXi host running Holodeck Toolkit. The Toolkit provisions: 1x HoloRouter (virtual router/gateway), 1x HoloConsole (jumpserver/webtop), 4x nested ESXi hosts (management cluster), 1x Cloud Builder, which in turn deploys: 1x SDDC Manager, 1x vCenter Server, 1x NSX Manager cluster (or single node depending on profile). All nested components communicate through the HoloRouter on an isolated 10.1.1.0/16 supernet.
graph TB PHY[Physical ESXi Host] --> HR[HoloRouter 10.1.1.1] PHY --> HC[HoloConsole/Webtop] HR --> ESXi1[ESXi-01 10.1.1.101] HR --> ESXi2[ESXi-02 10.1.1.102] HR --> ESXi3[ESXi-03 10.1.1.103] HR --> ESXi4[ESXi-04 10.1.1.104] HR --> CB[Cloud Builder 10.1.1.200] CB --> SDDC[SDDC Manager 10.1.1.5] CB --> VC[vCenter 10.1.1.6] CB --> NSX[NSX Manager 10.1.1.10]
IP Addressing
| Network | Purpose | VLAN |
|---|---|---|
10.1.1.0/20 | Management network (SDDC Manager, vCenter, NSX, Cloud Builder) | VLAN 1644 (default) |
10.1.1.0/24 | ESXi host management VMkernel | VLAN 1644 |
10.1.2.0/24 | vMotion | VLAN 1645 |
10.1.3.0/24 | vSAN | VLAN 1646 |
10.1.4.0/24 | NSX Host TEP | VLAN 1647 |
10.1.5.0/24 | NSX Edge TEP | VLAN 1648 |
Credentials
| System | Username | Password |
|---|---|---|
| Physical ESXi Host | root | Set during ESXi installation |
| HoloConsole (Webtop) | holodeck | Default Holodeck password from Toolkit docs |
| Nested ESXi Hosts | root | Set by Holodeck config.json esxiPassword field |
| Cloud Builder | admin@local | Set by Holodeck config.json cbPassword field |
| SDDC Manager | administrator@vsphere.local | Set in VCF bring-up spec JSON |
| vCenter Server | administrator@vsphere.local | Same as SDDC Manager SSO password |
| NSX Manager | admin | Set in VCF bring-up spec JSON |
Tasks
Task 1 Review and customize the deployment configuration
manageabilityUnderstand and validate the deployment parameters before committing to a 2+ hour automated build. In production, this mirrors the VCF Planning and Preparation Workbook review — a VCDX-critical checkpoint.
Open PowerShell on the HoloConsole (or your management workstation with the Holodeck module imported). Navigate to the Holodeck runtime directory: cd C:\Holodeck\holodeck-runtime
Open the deployment configuration file: notepad .\templates\config.json
Verify the vcfVersion field is set to '9.0.2' (or your target version). Verify contentManifest points to a valid manifest file for that version in the content/ directory.
Review ESXi host resource sizing in the managementCluster section. Default: 4 hosts, each with 12 vCPU and 96 GB RAM. If your physical host has less than 512 GB RAM, reduce to 3 hosts or lower per-host memory (minimum 64 GB per nested ESXi for management domain).
Verify the networking section: confirm the management CIDR (default 10.1.1.0/20), VLAN range (1644-1648), and that the holoRouterExternalIP is on an accessible network from your workstation.
Save and close config.json. Review the changes by running: Get-Content .\templates\config.json | ConvertFrom-Json | Format-List
Validation Gate
Check: Manually confirm: vcfVersion matches your target, contentManifest file exists, ESXi host count x resource per host < physical host capacity, VLAN range is clear.
Expected: All four checks pass. You are ready to prepare the instance.
Common Errors
Task 2 Prepare the Holodeck instance
availabilityThe Prepare phase stages all artifacts, validates host compatibility, and configures the HoloRouter. In production terms, this is the equivalent of the VCF Planning & Preparation phase — validating that infrastructure meets the deployment checklist before committing.
In PowerShell, run: New-HoloDeckInstance -ConfigFile .\templates\config.json -Verbose
Verify the HoloRouter is reachable: ping 10.1.1.1 from HoloConsole or your workstation (if routed).
Verify the HoloConsole webtop is accessible: open a browser and navigate to https://<holoconsole-ip>:8443
Check the Prepare output log: Get-Content C:\Holodeck\holodeck-runtime\logs\prepare-*.log -Tail 50
Validation Gate
Check: Run: Get-HoloDeckInstance | Format-Table -Property InstanceId, State, VcfVersion
Expected: Instance state shows 'Prepared' with your target VCF version
Common Errors
Task 3 Start the full VCF management domain deployment
availabilityThis is the automated bring-up — equivalent to running Cloud Builder in production. Holodeck orchestrates nested ESXi provisioning, Cloud Builder deployment, and the full VCF SDDC bring-up workflow. Understanding what happens inside this black box is critical for VCDX because panelists will probe your understanding of the bring-up sequence.
Run: Start-HoloDeckInstance -InstanceID <your-instance-id> -Verbose
While waiting, monitor the Cloud Builder UI for detailed bring-up progress. From HoloConsole webtop, open Firefox and navigate to https://10.1.1.200 (Cloud Builder). Log in with admin@local credentials from config.json.
Watch for the NSX validation step specifically. VCF 9.0.x requires NSX 9.0.2.x. If the deployment pauses with 'Validate NSX install image is available fails', check the content pack version.
Wait for the PowerShell command to complete. The final output should indicate success.
Validation Gate
Check: Run: Get-HoloDeckInstance | Format-Table -Property InstanceId, State, VcfVersion, ManagementDomainStatus
Expected: State: Running, ManagementDomainStatus: Active
Common Errors
Task 4 Validate management domain health — SDDC Manager and vCenter
manageabilityPost-deployment validation is where you prove the management domain is production-equivalent. VCDX panelists will expect you to articulate what 'healthy' looks like and what you would check. This task builds that muscle memory.
From HoloConsole webtop Firefox, navigate to SDDC Manager: https://10.1.1.5. Log in with administrator@vsphere.local.
Navigate to Inventory > Workload Domains. Verify the management domain is listed with Status: Active.
Navigate to Inventory > Hosts. Verify all 4 ESXi hosts show Status: Active with no warnings or errors.
Open a new tab. Navigate to vCenter: https://10.1.1.6. Log in with administrator@vsphere.local.
In vSphere Client, navigate to the management cluster. Verify: (a) 4 ESXi hosts are connected, (b) vSAN health is green, (c) DRS is enabled, (d) HA is enabled and admission control is configured.
Verify vCenter services: from vSphere Client, navigate to Administration > System Configuration > Services. Confirm all services are Running.
Validation Gate
Check: SDDC Manager: Workload Domains > management domain = Active. vCenter: Cluster > 4 hosts connected + vSAN green + HA/DRS enabled.
Expected: All management plane components healthy with no warnings
Common Errors
Task 5 Validate NSX Manager and transport node status
availabilityNSX is the most failure-prone component in nested deployments due to TEP tunnel overhead and resource contention. Validating NSX health independently is a VCDX best practice that demonstrates operational maturity.
From HoloConsole webtop, navigate to NSX Manager: https://10.1.1.10. Log in with admin and the NSX password from the bring-up spec.
Navigate to System > Fabric > Nodes > Host Transport Nodes. Verify all 4 ESXi hosts are listed with Configuration State: Success and Transport Node Status: Up.
Navigate to System > Fabric > Nodes > Edge Transport Nodes. Verify edge nodes are deployed (if your deployment profile includes them) with Status: Up.
Navigate to Networking > Tier-0 Gateways. Verify the default Tier-0 gateway is present and in Active status.
Run an NSX health check via API: From HoloConsole PowerShell, run: Invoke-RestMethod -Uri 'https://10.1.1.10/api/v1/cluster/status' -Headers @{Authorization='Basic '+[Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes('admin:<nsx-password>'))} -SkipCertificateCheck
Validation Gate
Check: NSX Manager: All host transport nodes Success/Up + cluster status STABLE
Expected: NSX fabric fully operational with no degraded components
Common Errors
Task 6 Configure external access and take a post-deployment snapshot
recoverabilityExternal access (reaching the nested environment from your workstation without the webtop) and a clean snapshot are operational hygiene steps that save hours of re-deployment time. This mirrors production change management: validate, document, snapshot.
On your physical workstation (outside the Holodeck pod), add a static route to reach the nested management network. Windows: route add 10.1.1.0 mask 255.255.240.0 <holorouter-external-ip> -p. Linux/Mac: sudo ip route add 10.1.1.0/20 via <holorouter-external-ip>
Verify external access: ping 10.1.1.5 (SDDC Manager) and ping 10.1.1.6 (vCenter) from your workstation.
Configure DNS forwarding on your workstation to resolve the nested environment's FQDNs. Add a conditional forwarder pointing *.site-a.vcf.lab to the HoloRouter DNS (10.1.1.1). Alternatively, add static hosts file entries for sddc-manager.site-a.vcf.lab (10.1.1.5), vcenter.site-a.vcf.lab (10.1.1.6), nsx-manager.site-a.vcf.lab (10.1.1.10).
Open your workstation browser and navigate to https://vcenter.site-a.vcf.lab (accept the self-signed certificate). Verify you can log in.
Take a snapshot of the entire Holodeck instance. In PowerShell on HoloConsole: snapshot all VMs in the management pod using: Get-VM -Name 'Holo-*' | New-Snapshot -Name 'holodeck-02-complete' -Description 'Post management domain deployment - all components healthy' -Memory:$false -Quiesce:$false
Validation Gate
Check: From workstation browser: (1) https://vcenter.site-a.vcf.lab loads, (2) https://sddc-manager.site-a.vcf.lab loads, (3) Get-VM -Name 'Holo-*' | Get-Snapshot shows holodeck-02-complete snapshot on all VMs.
Expected: External access working for all three management UIs + clean snapshot in place for all pod VMs
Common Errors
Final Validation
The management domain is fully deployed and externally accessible. All five management plane components (SDDC Manager, vCenter, NSX Manager, 4x ESXi hosts, Cloud Builder) are healthy. A clean snapshot is in place for future labs.
✓ SDDC Manager Dashboard: management domain Active → Active with 4 hosts, 1 cluster
✓ vCenter Cluster: 4 hosts connected, vSAN green, HA/DRS on → All green, no alarms
✓ NSX Manager: all host transport nodes Success/Up → 4/4 transport nodes healthy
✓ External access: FQDN-based UI access from workstation → Three management UIs reachable by FQDN
✓ Snapshot: holodeck-02-complete exists on all pod VMs → Snapshot present for rollback
Cleanup / Restore
Snapshot: holodeck-02-complete
• Snapshot all Holodeck VMs as 'holodeck-02-complete' (already done in Task 6)
• Close any open browser tabs to management UIs to free HoloConsole resources
• Document the IP-to-FQDN mapping in your lab notebook for reference in subsequent labs
Design Reflection (VCDX)
A VCDX panelist examining a VCF management domain design would probe: Why four hosts instead of three? What is the failure domain boundary? How does the management domain's availability model differ from workload domains? What are the implications of running SDDC Manager as a single instance (no HA)? How would you size the management cluster in a production environment where nested virtualization overhead doesn't exist? Be prepared to explain why the Holodeck configuration (4 ESXi hosts, single-fault vSAN) is a learning configuration, not a production reference architecture.
Requirements
- Management domain must host SDDC Manager, vCenter, NSX Manager, and provide compute for lifecycle operations
- All management services must be accessible from the operator workstation
- Environment must be restorable to a known-good state for repeatable lab exercises
Constraints
- Nested virtualization imposes ~15-25% performance overhead vs bare-metal
- Single physical host = single fault domain (no host-level HA in nested environment)
- Holodeck default resource sizing limits workload domain expansion
- Network isolation via HoloRouter prevents direct L2 adjacency with physical infrastructure
Assumptions
- Physical host has sufficient resources (512 GB RAM, 64 cores, 2 TB SSD)
- Internet connectivity is available for content downloads (or offline depot is pre-staged)
- VCF licensing is available (Holodeck provides evaluation licenses)
- Operator has basic PowerShell and vSphere administration skills
Risks
- Resource exhaustion on physical host causes cascading failures across all nested VMs — IMPACT: full lab loss, MITIGATION: snapshots + resource monitoring
- Holodeck Toolkit version incompatibility with VCF content pack — IMPACT: deployment failure, MITIGATION: pre-validate manifest checksums
- HoloRouter failure isolates entire nested environment — IMPACT: total management plane outage, MITIGATION: snapshot HoloRouter separately, know the FRR recovery procedure
- Nested vSAN performance may not represent production behavior — IMPACT: misleading performance baselines, MITIGATION: use vSAN health checks but don't benchmark nested I/O
Self-Assessment Discussion Prompts
- Why does VCF require a minimum of 4 hosts for the management domain? What would change with 3 hosts?
- The HoloRouter is a single point of failure. In production, what would replace its function? (Answer: physical ToR switches + BGP peering or static routes)
- If SDDC Manager becomes unavailable, can you still manage vCenter and NSX directly? What capabilities are lost?
- How does the VCF bring-up sequence enforce ordering (Cloud Builder -> SDDC Manager -> vCenter -> NSX)? Why can't these be deployed in parallel?
- What is the difference between vSAN FTT=1 (RAID-1) and FTT=0 in the context of a 4-node nested lab vs production?
- If you needed to add a workload domain to this environment, what additional resources and configuration would be required?
Extensions
Deploy a VCF 5.2.x Management Domain for Comparison
Repeat this lab using the VCF 5.2.x content manifest to experience the differences in bring-up workflow, component versions (vCenter 8.0, NSX 4.x), and SDDC Manager UI. Document 5 specific differences between the 5.2.x and 9.0.x deployment experiences.
sameDeploy with the -ManagementOnly Flag
Re-deploy using Start-HoloDeckInstance with -ManagementOnly to skip VCF Automation. Compare the resulting management domain to the full deployment. Identify which Day 2 operations require VCF Automation and which can be done via SDDC Manager alone.
sameDual-Site Holodeck Deployment
Configure a second Holodeck instance as Site B with cross-site networking (different IP ranges). This exercises the dual-site Holodeck configuration documented in the community thread 'Holodeck 9.0.2 dual-site IP config' and prepares you for VCDX stretched cluster and disaster recovery design discussions.
harderDeliberately Introduce and Recover from a Deployment Failure
Intentionally misconfigure one element (wrong NSX bundle version, insufficient memory, invalid VLAN) to trigger a known failure from the community KB. Practice interpreting the error logs, identifying root cause using the playbook, and recovering without a full redeploy.
harder⚠ Known Pitfalls (from Community KB)
References
- VMware Cloud Foundation 9.0 Planning and Preparation GuideTier 1 — Official
Official Broadcom documentation — the authoritative source for bring-up prerequisites, IP planning, and component sizing - VCF Holodeck Toolkit — Broadcom Community ForumTier 1 — Official
Broadcom-hosted community forum. 63 threads captured in our Holodeck KB (see holodeck-kb-normalized.json) - VCF Holodeck GitHub Repository — IssuesTier 1 — Official
Official issue tracker with 15 open issues as of April 2026 - William Lam — VCF Nested Lab DeploymentTier 3 — Expert Blog
William Lam's blog — authoritative community expert, VMware Staff Engineer. Covers Holodeck tips, nested VCF deployment, and automation techniques. - Cormac Hogan — VCF Storage and vSAN ArchitectureTier 3 — Expert Blog
Deep dives on vSAN within VCF — relevant to understanding storage tier health in the management domain