Holodeck Lab Setup & Operations
Master the VCF Holodeck Toolkit — your primary lab environment for all subsequent VCDX preparation. This section covers installation, deployment, troubleshooting, and ongoing management of nested VCF environments using the Holodeck Toolkit (a Broadcom-published, Fling-style toolkit for testing and training, not a supported product). Every other section in the Academy depends on a functioning Holodeck pod.
Version Evolution
Holodeck descends from the VCF 4.x-era 'VCF Lab Constructor' (the 9.x docs no longer use that name and state Holodeck 9.x is not a VMware supported product, similar to a Fling). Version 2.0 (released ~2024) added support for VCF 5.x with PowerShell-based orchestration.
The current generation is Toolkit 9.1 — primary for this section as of 24 Sep 2026; 9.0.2 remains installable and the 9.0.x guidance in this section stays authoritative for 9.0.x instances (DOCUMENTED: vmware.github.io/Holodeck versions.json lists 9.1 (alias latest), 9.0.2, 9.0.1, 9.0 — no 9.1.1 or 9.2; release/9.1 branch last commit 7 Aug 2026). Release dates: VCF 9.1.0 GA 12 May 2026 (techdocs release notes; the 9.1 upgrade guide prints 11 May 2026).
Holodeck 9.1.0: docs header 'May 2026'; Broadcom lifecycle portal GA date 17 Jun 2026 (portal export, 24 Sep 2026); GA announced on the VCF blog 1 Jul 2026. All three are first-party; the lifecycle portal is the release-management record. Prior 9.0-line releases, three in under a year: 9.0 (Jun 2025; lifecycle portal GA 23 Jun 2025), 9.0.1 (Oct 2025; portal 20 Oct 2025), 9.0.2 (docs Mar 2026; portal GA 17 Feb 2026; last 9.0-line build 9.0.2.19) — no End of Service is published for Holodeck. The 9.0.x line supports VCF 9.0.x and 5.2.x.
Key evolution: -Interactive parameter removed (Holodeck 9.0.2) and replaced by the Update-HoloDeckInstance cmdlet; -DeploySupervisor renamed to -DeploySupervisorWkldDomain; -InstanceID became mandatory; DNSMASQ split from monolithic pod to 3 DNS + 1 DHCP pods; Supervisor support expanded to management domain. Binaries are version-specific: each -Version reads its own /holodeck-runtime/bin/<version>/ folder (the ESX ISO plus the VCF Installer OVA for 9.x or the Cloud Builder OVA for 5.2.x).
The HoloRouter is a Photon OS VM running Kubernetes internally — all services (DNS via dnsmasq on 9.0.x — replaced by Technitium DNS from 9.1 per docs/index.md and maintainer Discussion #178, 15 Sep 2026 — DHCP, FRR routing with BGP, CoreDNS, webtop) run as K8s pods rather than as systemd services on the Photon OS host. Valid -Version values for the 9.1 toolkit (DOCUMENTED: 9.1 release notes + FAQ, read 24 Sep 2026): '9.1.0.0', '9.0.2.0', '9.0.1.0', '9.0.0.0', '5.2.4', '5.2.3', '5.2.2', '5.2.1', '5.2'.
The generic cmd_reference page still shows the older 9.0.2-era six-value list ('9.0.0.0', '9.0.1.0', '9.0.2.0', '5.2', '5.2.1', '5.2.2') — upstream documentation lag, not a second truth; confirm with Get-Help New-HoloDeckInstance on the live toolkit. 9.1.1.0 is not a valid -Version value as of 24 Sep 2026; no toolkit release for 9.1.1 has surfaced (VCF 9.1.1.0 GA 3 Sep 2026 per techdocs; maintainer advice on issue #151, 6 Jul 2026: greenfield with 9.1.0.0 binaries rather than patch versions). Download token required: VCF entitlement via purchased, trial, NFR, pass VCP, or VCP + VMUG Advantage.
Typical full deployment resources: ~30 CPU cores, ~325 GB RAM, ~1.1 TB disk. GitHub issues tracker: https://github.com/vmware/Holodeck/issues (27 open as of 24 Sep 2026 (GitHub) — 15 new issues filed since 1 Aug 2026 (#163–#177), of which 3 are closed; DNS reliability remains the #1 pain point, e.g. #164 'Webtop not resolving DNS with custom DNSDomain', open since 6 Aug 2026; the Apr 2026 count was 15). Triple-validated against: official docs, VMware blog GA announcements, live lab deployment, and PDF documentation.
Holodeck 9.1 (release dates as stated above) is the current major release: it doubles the VCF resource envelope, adds VNA distributed networking and GitOps deployment, rebuilds the HoloRouter service stack, and introduces six breaking changes to 9.0.x runbooks. No 9.0-to-9.1 upgrade path is documented — treat it as a clean redeploy. The holodeck91-* lab set is the parallel 9.1 twin of holodeck-01..09; the 9.0.x content in this section remains authoritative for 9.0.x instances.
Learning Outcomes
- Install and configure the Holodeck Toolkit on a physical ESXi host
- Deploy a complete VCF 9.0.x management domain in a nested environment
- Troubleshoot the top 10 most common Holodeck deployment failures using community-documented solutions
- Configure external access (routing, DNS) to the nested management domain
- Manage Holodeck instance lifecycle: snapshots, destroy, re-deploy, dual-site
- Articulate the differences between a nested lab environment and production VCF from a VCDX design perspective
- Track Holodeck version evolution (9.0 → 9.0.1 → 9.0.2) including parameter deprecations, renames, and new cmdlets
- Configure Offline Depot Appliance (ODA) for air-gapped deployments and understand binary management workflows
- Use Developer Mode environment variables for automated/CI-driven lab provisioning
- Monitor and work around active GitHub issues affecting Holodeck lab reliability
What is Holodeck and Why It Matters for VCDX#
The VCF Holodeck Toolkit is a Broadcom-published PowerShell/PowerCLI toolkit (shipped as a module inside the HoloRouter OVA) that deploys a complete, nested VMware Cloud Foundation environment on a single physical ESX host. It is not a VMware supported product — the 9.1 documentation calls it 'similar to a Fling', intended for testing and training environments, with issues raised on GitHub (vmware.github.io/Holodeck/9.1, read 1 Oct 2026). The 'VCF Lab Constructor' name belongs to the VCF 4.x-era predecessor and did not surface in the 9.x documentation.
It creates 4+ nested ESXi virtual machines, a HoloRouter for isolated networking (with a browser Webtop on TCP 30000), and orchestrates the full VCF bring-up sequence (VCF Installer → management domain: vCenter, NSX, VCF Operations and fleet management, SDDC Manager → optional VCF Automation; Cloud Builder was the 5.x-era bring-up appliance).
For VCDX candidates, Holodeck is indispensable because:
- Hands-on practice at scale: VCDX requires demonstrating deep operational experience with VCF. Holodeck provides a safe, repeatable environment for every deployment, upgrade, and troubleshooting scenario you'll face in the exam.
- Design validation: Before defending a VCDX design, you can prototype it in Holodeck — test topology choices, failure scenarios, and scaling limits.
- Failure injection: Holodeck deployments frequently encounter real-world failure modes (networking, resource, versioning) that mirror production issues. Learning to troubleshoot these builds the operational instinct panelists are looking for.
- Version comparison: With Holodeck, you can run VCF 5.2.x and 9.0.x side-by-side on the same hardware, understanding the architectural evolution — a key VCDX discussion topic.
Key Takeaways
- Holodeck is your primary lab vehicle for all VCDX preparation — treat it as critical infrastructure
- Nested environments have real limitations (performance overhead, single fault domain) that you must articulate in VCDX discussions
- The Broadcom community forum (63 threads) and GitHub issues are your troubleshooting lifeline; resolutions are captured in each lab's Common Errors section
Holodeck Architecture Deep Dive#
Understanding Holodeck's architecture is essential because every component maps to a production VCF concept:
HoloRouter — A Photon OS VM running Kubernetes internally. All services run as K8s pods (NOT systemd): dnsmasq-dns (DNS with ~160 pre-provisioned records), dnsmasq-dhcp (DHCP for management and TEP VLANs), FRR (BGP routing with AS 65000 for router, AS 65001/65002 for edge domains), CoreDNS, and webtop (browser-based access). It provides L3 routing via 16+ VLAN subinterfaces (eth1.10 through eth1.25) between the nested environment and the physical host, with VRF 'sitea' isolating NSX Edge uplink VLANs.
On the default Site A master CIDR 10.1.0.0/20 the HoloRouter answers at 10.1.1.1 on the Untagged-HoL management subnet (Get-HoloDeckSubnet -Site a); external access uses the HoloRouter's own external management IP set at OVA deployment (SSH 22, Webtop 30000). In production, this function is served by physical ToR switches and datacenter core routers. The HoloRouter is the most critical single point of failure in the nested environment.
IMPORTANT: the HoloRouter network services (dnsmasq DNS/DHCP, FRR, webtop) run as Kubernetes pods — manage them with kubectl get pods / logs / exec, not with systemctl (the appliance itself is a Photon OS host).
Webtop — Browser-based desktop access to the nested environment, served by the HoloRouter on TCP 30000 (http://<holorouter-ip>:30000; in 9.0.2 it has no authentication and the docs say not to expose it externally; Holodeck 9.1 fronts it at https://webtop.vcf.lab with the documented admin credentials). There is no separate HoloConsole VM in Holodeck 9.x. In production this maps to a jump host or bastion server in the management network.
Nested ESXi Hosts — VMs running ESXi, configured with virtual NVMe disks (for vSAN), multiple VMkernel adapters (management, vMotion, vSAN, TEP), and nested virtualization enabled. These behave identically to physical hosts from VCF's perspective, with the exception of performance characteristics.
VCF Installer (Cloud Builder for 5.2.x) — The appliance that runs the VCF 9.x bring-up: it is deployed by Holodeck from /holodeck-runtime/bin/<version>/, downloads the remaining VCF components from the online depot (Broadcom build token) or an offline depot, and deploys the management domain (vCenter, NSX, VCF Operations and fleet management, SDDC Manager, optionally VCF Automation). The VCF Installer appliance replaces the Cloud Builder appliance in VCF 9.0 (VCF 9.0 release notes).
It ships in the same OVA as SDDC Manager (VCF-SDDC-Manager-Appliance-9.x.ova): deployed inside the management infrastructure, the appliance switches into SDDC Manager mode after the workflow and cannot be used as an installer again (VCF 9.0.1 docs p.1265). Holodeck deploys the installer (vcfinstaller-a) as its own VM on the target host beside the nested ESX VMs, on the same trunk port group, and the bring-up deploys a separate SDDC Manager (sddcmanager-a); Remove-HoloDeckInstance deletes the nested ESX hosts and the VCF Installer VM together.
Holodeck uses Cloud Builder only for -Version 5.2.x targets.
Binaries per version — Holodeck needs exactly two files staged on the HoloRouter in /holodeck-runtime/bin/<version>/: the ESX ISO and the VCF Installer OVA (Cloud Builder OVA for 5.2.x). The pre-check '<Component> binary was not found' fails the deployment until both are present; there is no Academy-style content manifest in the documented workflow (the only bill-of-materials file is /holodeck-runtime/templates/bom.json, used by a 9.0.1 NSX image workaround).
Mermaid source
graph TD
subgraph PH["Physical host"]
ESX["Target ESX 8.0 U3 or 9.0 host, or vCenter cluster"]
DS[("SSD/NVMe datastore 1.1 TB+")]
PG["Trunk port group VLAN 4095, MTU 9000, security Accept x3"]
end
subgraph HD["Holodeck 9.0.2 instance - Site A 10.1.0.0/20, VLANs 0 and 10-25"]
HR["HoloRouter: DNS, DHCP, NTP, L3 routing, BGP, Webtop :30000, pwsh + Holodeck module"]
N1["Nested ESX esx-01a"]
N2["Nested ESX esx-02a"]
N3["Nested ESX esx-03a"]
N4["Nested ESX esx-04a"]
VI["VCF Installer vcfinstaller-a"]
end
subgraph MD["Management domain VMs"]
VC["vCenter vc-mgmt-a"]
NSX["NSX Manager nsx-mgmt-a"]
OPS["VCF Operations ops-a + fleet management"]
SDDC["SDDC Manager sddcmanager-a"]
end
ESX --> HR
ESX --> N1
ESX --> N2
ESX --> N3
ESX --> N4
HR -->|New-HoloDeckInstance| VI
VI --> VC
VI --> NSX
VI --> OPS
VI --> SDDC
N1 -->|vSAN| DS
N2 -->|vSAN| DS
N3 -->|vSAN| DS
N4 -->|vSAN| DSVersion Comparison
VCF 5.2.x
Holodeck for VCF 5.2.x uses the same cmdlets with -Version 5.2.x and stages the ESX ISO plus the Cloud Builder OVA in /holodeck-runtime/bin/5.2.x/ instead of the VCF Installer OVA. NSX version is 4.x instead of 9.x. SDDC Manager UI and API differ significantly. The deployment workflow is identical in structure.
VCF 9.0.x
VCF 9.0.x introduced consolidated NSX + VCF versioning (NSX 9.0.x aligns with VCF 9.0.x). The -ManagementOnly and -DeployVcfAutomation flags are new. Dual-site deployment support was added.
Key Takeaways
- Every Holodeck component has a production equivalent — learn to translate between lab and production contexts
- The HoloRouter is the single point of failure; know how to troubleshoot FRR, DNS, and DHCP on it
- Binary staging is non-negotiable: the ESX ISO and the VCF Installer OVA (Cloud Builder for 5.2.x) must sit in /holodeck-runtime/bin/<version>/ for the exact -Version you deploy; a missing or mismatched binary fails the pre-check
Community Knowledge Base — Top Deployment Issues#
The Holodeck community knowledge base below is drawn from public sources:
Source 1: Broadcom Public Forums — 63 threads, 247 posts. These are publicly searchable and tend toward initial setup/deployment questions.
Source 2: GitHub — github.com/vmware/Holodeck issues and discussions — public bug reports, workarounds and maintainer answers for the toolkit itself (for example the maintainer answer in GitHub Discussion #178 on the one-HoloRouter-per-instance model).
What the public reports show:
Most reports concern the Deploy / Bring-up phase — getting the management domain up is where most users struggle.
Top failure categories (community reports — Broadcom community forum and GitHub issues):
- Datastore selection failures (physical host storage configuration)
- Binary or depot version mismatches (VCF Installer version vs the VCF version deployed, NSX image validation — e.g. GitHub issue #53)
- HoloRouter configuration failures (DHCP, FRR, port group VLAN trunking, adapter B/A swap — e.g. GitHub issue #29, 'Stuck - Waiting for FRR Service')
- Offline depot authentication bugs (disable auth on HTTP, manual VCF Installer config — e.g. GitHub issue #60)
- DNS resolution problems on the HoloRouter (check the HoloRouter DNS service and upstream forwarders)
- Physical host resource sizing (check the Holodeck documentation's resource tables for your deployment type)
- Legacy CPU compatibility (boot.cfg allowLegacyCPU flag)
- Certificate failures (SDDC Manager VMCA /etc/hosts fix — GitHub issues #42 and #43; NSX signing key expiry KBs)
- State file manipulation for skipping/retrying failed deployment steps
- One-config-per-deployment rule and ProvisionOnly stickiness
25 of 63 forum threads have confirmed resolutions.
Resolutions are captured in each lab's Common Errors section with links to the public source (forum thread, GitHub issue or release-notes known issue).
Key Takeaways
- Pre-validate the staged binaries, the depot (online token or offline depot) and the physical host resources BEFORE running New-HoloDeckInstance — this removes the most common causes of deployment failure listed above
- If stuck, check the lab's Common Errors section and the public forum and GitHub issues first — your issue may already be documented with a resolution
- Each lab's Common Errors section carries the exact commands, file paths and step-by-step fixes, with a link to the public source
- Keep the community forum bookmarked — it's updated regularly with new issue reports and solutions
Holodeck 9.1 Architecture and Support Matrix (9.1)#
Holodeck 9.1 is a major toolkit release aligned to the VCF 9.1 GA train, and it resets the baseline assumptions an architect makes about the nested lab platform. Release dates: VCF 9.1.0 GA 12 May 2026 (techdocs release notes; the 9.1 upgrade guide prints 11 May 2026). Holodeck 9.1.0: docs header 'May 2026'; Broadcom lifecycle portal GA date 17 Jun 2026 (portal export, 24 Sep 2026); GA announced on the VCF blog 1 Jul 2026. All three are first-party; the lifecycle portal is the release-management record.
Deployment targets and support matrix. The 9.1 toolkit deploys VCF 9.1.0.0, VVF 9.1.0.0, and the VCF 5.2.4 maintenance line (official docs additionally list 5.2.3, which the community announcement omitted). The full supported deployment range spans VCF 9.1.0.0 down to VCF 5.2, and the release notes give a minimum ESX version of 8.0 U3; the docs list vSphere 8.0 U3 and 9.0 as the tested target (physical) host versions. Nested ESX versions follow the VCF release being deployed. VCF Installer pairing is version-sensitive: Installer 9.1.0.0 is supported for 9.1 deployments, Installer 9.0.2.0 is recommended for the 9.0.x line, and Installer 9.0.0.0 is no longer supported at all. Two product lines continue to coexist: Holodeck 5.2x (VCF 5.2.x only) and Holodeck 9 (5.2.x plus 9.x).
Resource envelope. Requirements step up substantially versus the 9.0 baseline (~30 cores / ~325 GB / ~1.1 TB typical). Published VCF 9.1 figures: 48 CPU cores, 650 GB RAM, 2.2 TB disk for single site, doubling to 96 cores / 1.5 TB / 5 TB for dual site; VVF 9.1 needs 24 cores / 512 GB / 2 TB. New in 9.1, soft validation lets pre-checks warn rather than block when resources fall below recommendation — operationally convenient on constrained shared hosts, but the architect explicitly owns the resulting performance risk. The HoloRouter appliance itself grows to 6 vCPU / 12 GB (from 4 vCPU / 8 GB) to carry its expanded services stack, and no in-place 9.0-to-9.1 router upgrade is documented — plan a clean 9.1 HoloRouter deployment until Broadcom says otherwise.
Design significance. The wide version range turns upgrade-path rehearsal into a first-class lab scenario: one toolkit can stand up a 5.2.4 instance beside a 9.1.0.0 instance and make the 5.2-to-9.x architectural evolution (Cloud Builder vs VCF Installer, NSX 4.x vs 9.x-aligned versioning, licensing and lifecycle changes) a demonstrable exercise rather than a whiteboard claim. What did not change matters equally: a single physical host remains a single fault domain, default networking carries forward (Site A 10.1.0.0/20, Site B 10.2.0.0/20, /20 MasterCIDR rule), and the one-config-one-deployment discipline persists. The 9.0.x guidance in this section remains authoritative for 9.0.x instances; the holodeck91-* lab set is its parallel 9.1 twin, to be enriched during your own live 9.1 deployment.
Key Takeaways
- Targets: VCF 9.1.0.0 / VVF 9.1.0.0 / VCF 5.2.4; supported range VCF 9.1.0.0 down to 5.2; minimum physical target-host ESX 8.0 U3
- Resource floor rises to 48 CPU / 650 GB / 2.2 TB (VCF 9.1 single site); soft validation warns instead of blocking below-spec deploys — you own the performance risk
- HoloRouter grows to 6 vCPU / 12 GB with no documented in-place upgrade — plan a clean 9.1 router deployment
- The 9.1.0.0-to-5.2 range makes side-by-side upgrade rehearsal a first-class lab scenario
VNA Distributed Cluster Networking vs the Single-Router Model (9.1)#
The headline architectural change in Holodeck 9.1 is Virtual Network Appliance (VNA) cluster support for both management and workload domains — a distributed alternative to the single-HoloRouter networking model that has defined every prior Holodeck generation.
The 9.0-era model and its limits. Through 9.0.x, the HoloRouter was the entire network fabric: L3 routing across all VLAN sub-interfaces, DNS, DHCP, NTP, firewall, and the BGP (FRR) peering that NSX Tier-0 gateways depend on. That concentration made it the most critical single point of failure in the nested environment — one appliance failure takes down routing, name resolution, and edge peering for every site simultaneously. It also imposed a scale ceiling: every deployment's north-south path transits one appliance, and edge/transport placement cannot be meaningfully modeled because there is only one place for it to live.
What VNA changes. With 9.1, New-HoloDeckInstance gains -VnaClusterMgmtDomain and -VnaClusterWkldDomain switches that deploy VNA clusters as the networking layer for the respective domain. Supervisor deployment modes are wired to this choice: Centralized mode rides an NSX Edge cluster, while Distributed mode rides a VNA cluster (the -DeploySupervisorMgmtDomain / -DeploySupervisorWkldDomain parameters changed from switches to strings taking 'Centralized' or 'Distributed'). A 9.1.0.0 deployment reported in GitHub issue #143 combined -NsxEdgeClusterMgmtDomain and -VnaClusterMgmtDomain in a single management-only run, so the models can coexist in one instance.
Availability and scalability design reading. For a VCDX candidate this is precisely the centralized-vs-distributed services argument panels probe in production designs. A distributed cluster spreads the networking function across appliances, shrinking blast radius and removing the single choke point — at the cost of more moving parts, more capacity consumed, and a more complex failure-mode analysis (partial cluster degradation instead of clean binary loss).
The lab now lets you rehearse both: model a centralized edge design and a distributed VNA design on identical nested hardware and defend the trade-off with observed behavior instead of doctrine. Note that the HoloRouter itself does not disappear — core infrastructure services (DNS via Technitium, DHCP, BGP, the reverse-proxied service stack) still live there, so the fabric-services fault domain and the domain-networking fault domain must now be analyzed separately.
Open questions. VNA cluster sizing and host-count minimums were not published in the Holodeck 9.1 release notes or documentation when checked, and the incremental resource overhead is unquantified — both are items to capture during your own live 9.1 deployment. Until then, treat VNA sizing as unverified and budget conservatively inside the 9.1 resource envelope.
Key Takeaways
- VNA clusters (-VnaClusterMgmtDomain / -VnaClusterWkldDomain) provide a distributed alternative to the single-HoloRouter model for both domain types
- Supervisor modes map to the networking choice: Centralized = NSX Edge cluster, Distributed = VNA cluster (string-typed parameters, new in 9.1)
- Distributed networking shrinks blast radius and removes the single choke point at the cost of capacity and failure-mode complexity — a rehearsable VCDX trade-off
- VNA sizing and host minimums are unverified — capture during your own live 9.1 deployment
GitOps Deployment from the GitLab UI vs the PowerShell Path (9.1)#
Holodeck 9.1 introduces a second deployment driver alongside the classic PowerShell cmdlets: GitOps deployment from the GitLab UI. When the HoloRouter OVA is deployed with GitOps enabled (the default), GitLab is automatically installed and configured on the HoloRouter, a repository pre-loaded with the Holodeck deployment pipeline is created, and the HoloRouter registers itself as a GitLab runner with the appropriate tags — letting you trigger a complete Holodeck deployment directly from the GitLab UI with no manual PowerShell.
What each path offers. The PowerShell path is imperative: New-HoloDeckConfig, then New-HoloDeckInstance with explicit switches, verbose console output, and fine-grained control at every step. Its strengths are visibility and intervention — you see each phase, can resume idempotently from failure, and can compose custom automation around the cmdlets. The GitOps path is declarative-by-pipeline: deployment logic lives in a versioned Git repository, execution happens through pipeline runs, and every run leaves a durable record.
Its strengths are repeatability (the pipeline is the same every time; configuration drift between operators disappears) and auditability (pipeline history answers who deployed what, when, with which commit — a change-control story that ad-hoc PowerShell sessions cannot match). For a team lab or any environment where deployments must be reconstructable, that audit trail is the architecturally significant feature, not the convenience.
Interactions and caveats. The cmdlets remain fully available — GitOps is additive, not a replacement — and even GitOps-driven labs still use the PowerShell module for queries and Day-2 operations. Practical caveats at release: a first-boot issue (GH#139) can leave GitLab unreachable until the iptables port 80/443 ACCEPT rules are moved above the DROP rule; the bundled GitLab version and its resource overhead are unpublished; and on a multi-tenant shared host, a UI-triggered full deployment is exactly the kind of high-blast-radius action that needs coordination rules before anyone clicks it. 9.1 also handles build tokens and credentials as secure strings, improving the hygiene of both paths.
Architect framing. This is a repeatability/auditability-versus-control trade-off you should be able to argue both ways in a defense: choose GitOps when the priority is standardized, reviewable, reconstructable deployments across a team; choose PowerShell when the priority is granular control, interactive troubleshooting, or integration into existing automation. Document the choice and its rationale per environment — the deliberateness is what a panel is probing, not the tool.
Key Takeaways
- A GitOps-enabled HoloRouter auto-provisions GitLab with a pre-loaded pipeline and registers itself as the runner — full deployments trigger from the UI
- PowerShell = imperative control and visibility; GitOps = repeatability plus a pipeline-history audit trail; the cmdlets remain fully supported either way
- First-boot caveat GH#139: reorder iptables 80/443 rules if GitLab is unreachable; bundled GitLab version and overhead are unverified
- Defend the driver choice as a deliberate repeatability/auditability-vs-control decision, made per environment
The 9.1 HoloRouter Services Stack — and What It Does Not Yet Supersede (9.1)#
Holodeck 9.1 rebuilds the HoloRouter's service surface into an integrated, secured stack, reachable via the router's management IP behind an HTTPS reverse proxy with automated certificate management (an internal CA at ca.vcf.lab, with certificates auto-provisioned by a Vault backend).
The stack. HashiCorp Vault (:8200, vault.vcf.lab) provides secrets management and automated certificate provisioning for the other services. Authentik SSO (:9443, auth.vcf.lab) is a full identity provider — OIDC, OAuth2, SAML 2.0, and SCIM 2.0 provisioning — and 9.1 ships two new cmdlets, Initialize-Authentik and Set-VCFSSOConfiguration, that bootstrap an OIDC provider and wire VCF Operations to it with SCIM sync, giving the lab a production-shaped SSO chain. Technitium DNS (:5380, dns.vcf.lab) becomes the DNS management surface: zones, records and DHCP scopes are managed through the Technitium web UI, and the Get-/Set-/Remove-HoloDeckDNSConfig cmdlets are deprecated from 9.1 (valid only up to and including 9.0.2). Webtop (:30000, webtop.vcf.lab) — the browser desktop — is now behind authentication, and with GitOps enabled, GitLab and its container registry join the stack.
Infrastructure services always live on the vcf.lab domain regardless of any custom -DNSDomain; only nested VCF component FQDNs follow the custom domain.
Two caveats on 'HTTPS everywhere'. The claim is real but not absolute. Per the docs, "the individual services run on HTTP internally but are exposed securely via HTTPS at the FQDNs above" — and Webtop on http://<ip>:30000 explicitly bypasses the reverse proxy.
Separately, the Authentik username is documented inconsistently: the 9.1 release notes say admin, while the Getting Started → "Holodeck Environment Overview" page says akadmin (the 9.1 docs carry this on Getting Started, not on any separate architecture reference — cite Getting Started). akadmin is the Authentik product default, but the two names may simply be two different accounts: the 20 Jul 2026 VCF blog "Simplified Identity Management with Authentik for Holodeck 9.1" (https://blogs.vmware.com/cloud-foundation/2026/07/20/identity-mgmt-for-holodeck/), which documents Initialize-Authentik and Set-VCFSSOConfiguration, names only admin@vcf.lab — the VCF Operations SSO user — so akadmin may be the Authentik local superuser and admin@vcf.lab the SSO identity it provisions (ASSUMED, 24 Sep 2026; settled by a live login on first deploy).
The item stays open with that framing.
What Technitium supersedes — the DNS question, settled (DOCUMENTED, 24 Sep 2026). An earlier revision of this section read Technitium as retiring dnsmasq outright; the revision of 31 Aug 2026 walked that back to a working hypothesis — that Technitium was only a management UI layered over a still-present dnsmasq layer, with the 9.0-era fixes therefore carried forward — because the cmd_reference page still describes Set-HoloRouter as configuring "DNSMASQ for DNS services, FRR for BGP routing". That hypothesis is contradicted by every maintained source.
The Getting Started page (docs/index.md, branch release/9.1) states "Starting with Holodeck 9.1, DNS management is handled via Technitium DNS"; the GA blog (1 Jul 2026) says "Holodeck 9.1 replaces dnsmasq with Technitium as an advanced, web-managed DNS server"; the 9.1 release notes carry the bug-fix line "Fixed DNS lookup and configuration issues with Technitium DNS integration"; and a maintainer answer in GitHub Discussion #178 (https://github.com/vmware/Holodeck/discussions/178, 15 Sep 2026) confirmed it: "Only change in Holodeck 9.1 in this regard is we replaced dnsmasq with technitium." The Set-HoloRouter DNSMASQ one-liner is the only remaining mention, and it sits on the unmaintained cmd_reference page (upstream lag, the same page whose -Version list is stale).
Consequences for this section: the /holodeck-runtime/dnsmasq/dnsmasq_configmap.yaml upstream-forwarder workaround is 9.0.x-only and does not carry to 9.1; still supply upstream DNS as an IP address, never an FQDN — that remains recommended practice even though its original rationale (an FQDN stored as 1.1.1.1#cloudflare-dns.com, with dnsmasq reading # as a port separator) is dnsmasq-specific; and the 9.1-era DNS defect to track is issue #164 "Webtop not resolving DNS with custom DNSDomain" (open, 6 Aug 2026).
Residual (ASSUMED): whether any dnsmasq pod still runs under the 9.1 HoloRouter — settled by kubectl get pods -A on a live 9.1 router.
Multi-instance rule — one HoloRouter per instance (DOCUMENTED, Discussion #178, 15 Sep 2026). The same maintainer answer fixes the shared-host model: "One HoloRouter per Holodeck instance is required as the networking set up on Holorouter is for one specific nested VCF instance.
Since each instance is on its own HoloRouter, their networking is setup independently from each other from scracth." For several instances on one physical ESX host, pass -VLANRangeStart to New-HoloDeckInstance so each instance runs on its own non-overlapping VLAN set; if NSX is deployed on the physical estate, put each HoloRouter on its own NSX segment.
The 9.0-era caution that Set-HoloRouter is global and destructive to whatever lives on that router still describes the appliance, but the correct mitigation is now structural rather than procedural: never share one HoloRouter between instances, so Set-HoloRouter is a per-instance concern and not a cross-tenant one.
Design reading. The security posture shift — HTTPS everywhere, authenticated access, centralized secrets, real SSO — moves the lab appreciably closer to a production trust model, which strengthens the security dimension of any lab-derived design narrative. It also concentrates still more critical services on one appliance: the availability analysis of the HoloRouter as a fault domain gets heavier in 9.1, not lighter. And the epistemic lesson is itself defensible material: an architect who says "one unmaintained reference page disagreed with the maintained docs, so I checked the maintainers' answer and re-baselined it in the lab before trusting it" is demonstrating exactly the rigour a panel is testing for.
Version Comparison
VCF 9.0.x
HoloRouter runs dnsmasq (split into 3 DNS pods + 1 DHCP pod in 9.0.2), FRR for BGP, and an unauthenticated Webtop on :30000 as loosely coupled Kubernetes pods over open HTTP. DNS records managed via Get-/Set-/Remove-HoloDeckDNSConfig.
VCF 9.1
Vault, Authentik SSO, Technitium DNS, internal CA and optional GitLab are added as OVA-resident services behind an HTTPS reverse proxy; Webtop requires auth; DNS cmdlets deprecated in favour of the Technitium UI. Technitium replaces dnsmasq (docs/index.md; GA blog 1 Jul 2026; maintainer on Discussion #178, 15 Sep 2026) — the 9.0.x dnsmasq forwarder fix does not carry forward; one HoloRouter per instance, -VLANRangeStart separates instances on one host.
Key Takeaways
- Stack: Vault :8200, Authentik SSO/SCIM :9443, Technitium DNS :5380, authenticated Webtop :30000 — behind an HTTPS reverse proxy with an internal CA at ca.vcf.lab
- Initialize-Authentik + Set-VCFSSOConfiguration build a production-shaped OIDC/SCIM SSO chain into VCF Operations
- Technitium replaces dnsmasq in 9.1 — DOCUMENTED by docs/index.md (Getting Started), the GA blog of 1 Jul 2026, the 9.1 release-notes bug-fix line, and maintainer confirmation on GitHub Discussion #178 (15 Sep 2026); the lone DNSMASQ mention left is the Set-HoloRouter one-liner on the unmaintained cmd_reference page
- The dnsmasq_configmap.yaml forwarder fix is 9.0.x-only; still give upstream DNS as an IP, never an FQDN; one HoloRouter per instance, with -VLANRangeStart (and optionally per-router NSX segments) for multi-instance hosts; residual: confirm no dnsmasq pod on a live 9.1 router with kubectl get pods -A
- 'HTTPS everywhere' has exceptions: services are HTTP internally, and Webtop :30000 bypasses the proxy entirely
Day-2 Operations In-Toolkit: Hosts, Automation, Supervisors (9.1)#
Through 9.0.x, Holodeck was overwhelmingly a bring-up tool: Day-0/Day-1 deployment was automated, but growing or extending a running instance meant either full redeployment or manual surgery. Holodeck 9.1 changes the platform's character by making Day-2 lifecycle operations first-class: Update-HoloDeckInstance expands from two to five parameter sets, each an exclusive operation against a running deployment.
The five operations. (1) -AdditionalCluster adds a three-node vSphere cluster to the management or a workload domain. (2) -AddVcfAutomationAllAppsOrg configures the VCF Automation All-Apps organization on an existing VCFA deployment. (3) -DeploySupervisor stands up a Supervisor in the management or workload domain Day-2, with -SupervisorDeploymentMode selecting 'Centralized' (NSX Edge-backed) or 'Distributed' (VNA-backed, VCF 9.1.0.0+ only; the 9.1 docs list both options and state no default).
(4) -DeployVcfAutomation adds VCF Automation post-deploy through VCF Operations fleet management — the documented pattern is deliberate: deploy the rest of the stack first, verify capacity, then add VCFA. (5) -DeployNewHosts adds nested ESX hosts to a live deployment with explicit -CPU, -MemoryInGb, -Nodes, -vSANMode, and optional per-disk sizing; the older New-HoloDeckESXiNodes cmdlet is retained alongside it, and the relationship between the two paths is a live-verification item.
Why this matters architecturally. First, it keeps Day-0 short and lean: instead of front-loading a 12-hour full-stack deployment, you bring up a minimal management domain and stage capacity-hungry components (VCFA, Supervisors, extra clusters) incrementally, checking headroom at each step — exactly the staged-consumption discipline a capacity-constrained production rollout uses. Second, it makes expansion and lifecycle scenarios rehearsable: add-host workflows, cluster growth, and Supervisor enablement become repeatable lab exercises rather than redeploy-everything events, which pairs directly with the VKS 3.6 content elsewhere in the Academy. Third, for a VCDX defense it supplies operational evidence for Day-2 claims — you can demonstrate the expansion path you designed, not just assert it.
Caveats. Parameter sets are exclusive (one operation per invocation); everything is site-scoped via the mandatory -Site parameter. Known issue GH#143 documents a Day-2 All-Apps Org failure on fresh management-only deployments (an IP Space vs gateway-CIDR mismatch, with a community workaround), and 9.1-specific timings for Day-2 operations are unpublished — capture actuals during your own live 9.1 deployment before quoting maintenance-window numbers in any design artifact.
Key Takeaways
- Update-HoloDeckInstance now has five exclusive Day-2 parameter sets: AdditionalCluster, AddVcfAutomationAllAppsOrg, DeploySupervisor (Centralized/Distributed), DeployVcfAutomation, DeployNewHosts
- Staged consumption becomes the norm: keep Day-0 minimal, verify capacity, then add VCFA / Supervisors / hosts incrementally
- Distributed Supervisor mode requires the VNA path and VCF 9.1.0.0+; Day-2 operations are site-scoped and one-per-invocation
- GH#143 (All-Apps Org IP Space mismatch) has a community workaround; Day-2 timings are unverified until your own live 9.1 deployment
Holodeck Deployment States
26 named states representing progressive lab configurations. Each state is a checkpoint you can snapshot and restore.
State Reference
VCF 5.2 Standard Path
VCF 9.x Standard Path
VCF 9.x Advanced Configurations
Deployment Variants
Utility States
Exam Mapping: VCDX-VCF Architect — VMware Certified Distinguished Expert — VCF Architect
- Design VCF management domain architecture
- Define management cluster sizing and resource allocation
- Design network architecture for management and workload domains
- Articulate availability and recoverability for management plane components
Labs in This Section
References
- Holodeck 9.1 documentation (Getting Started)Tier 1 — Official
- Holodeck Release NotesTier 1 — Official
- Announcing the General Availability of Holodeck 9.1 (VMware Cloud Foundation blog, 1 July 2026)Tier 1 — Official
- VCF 9.0 documentation (for the 9.0.x guidance in this section)Tier 1 — Official
- VCF Holodeck Toolkit — Broadcom Community ForumTier 2 — Supporting source
- VCF Holodeck GitHub RepositoryTier 1 — Official
- Community blog — VCF Nested Lab Deployment SeriesTier 3 — Background reading
- Holodeck GitHub Discussion #178 (maintainer answer: one HoloRouter per instance; Technitium replaces dnsmasq)Tier 2 — Supporting source