Academy/Holodeck Lab Setup & Operations/Dual-Site Auto-Generation on Holodeck 9.1 — Two-File Site A + Site B Configuration and Second-Site Deployment
This lab targets VCF 9.1.0.0

Dual-Site Auto-Generation on Holodeck 9.1 — Two-File Site A + Site B Configuration and Second-Site Deployment

VCF 9.1.0.0Advancedarchitectvcdx⏱ 180 min

Holodeck 9.1 supports deployment of VCF 9.1.0.0, VVF 9.1.0.0, and the VCF 5.2.4 maintenance line (full range VCF 9.1.0.0 down to 5.2). This lab targets a VCF 9.1.0.0 dual-site deployment. 9.0.x twin: holodeck-06 (Dual-Site Holodeck Deployment) — 9.1 keeps the per-site New-HoloDeckNetworkConfig / Set-HoloRouter -dualsite sequence and adds the two-config-file New-HoloDeckConfig and the -Site parameter, so that lab remains valid background reading. Do NOT mix 9.0.x and 9.1 toolkit modules in the same PowerShell session.

What changed in 9.1:

Holodeck 9.1 changes the dual-site workflow: New-HoloDeckConfig now creates both a Site A and a Site B config file in one pass (Site A loaded; Site B loaded later with Import-HoloDeckConfig -ConfigId <id> -Site b), to eliminate race conditions when the sites are deployed in parallel (9.1 GA blog). Networking is still prepared per site in the documented order: New-HoloDeckNetworkConfig -Site a -MasterCIDR 10.1.0.0/20, New-HoloDeckNetworkConfig -Site b -MasterCIDR 10.2.0.0/20 (add -VLANRangeStart for custom ranges), Set-HoloRouter -dualsite (one VRF per site), then New-HoloDeckInstance -Site a and, from a second PowerShell tab, New-HoloDeckInstance -Site b (Holodeck 9.1 docs, Dual Site Deployment). 9.1 also fixes the VLAN-range handling and Site B port-group defects of 9.0.x (Holodeck 9.1 GA announcement).

Objectives

  • Generate the dual-site (Site A + Site B) Holodeck configuration files with one New-HoloDeckConfig call, prepare each site's networking with New-HoloDeckNetworkConfig and Set-HoloRouter -dualsite, and explain what 9.1 changed from the 9.0.x workflow
  • Inspect and verify the auto-generated Site A and Site B configuration files for non-overlapping CIDR, VLAN, and port-group allocation
  • Switch the active PowerShell session context between site configurations using Import-HoloDeckConfig with the 9.1 -Site parameter
  • Run pre-deployment validation for both sites and deploy the Site B management domain alongside an existing Site A instance
  • Validate cross-site reachability and management-plane health across the dual-site topology
  • Articulate, in VCDX defense terms, why automated configuration generation reduces configuration drift, and defend the choice between dual-site DR and stretched-cluster topologies

Prerequisites

Holodeck 9.1 tooling installed and a Site A VCF 9.1.0.0 management domain deployed and healthy (holodeck91-03 completed), OR a clean physical host if you intend to deploy both sites in this lab. Physical host sizing for dual-site VCF 9.1 (Holodeck 9.1 docs, recommended): 96 CPU, 1536 GB RAM, 5 TB SSD/NVMe — single-site VCF 9.1 is 48 CPU / 650 GB / 2.2 TB, so Site B needs roughly 650 GB RAM and 2.2-2.8 TB of additional disk; soft validation warns below this. The physical target host must be ESX 8.0 U3 or 9.0; nested hosts run the ESX build staged for the VCF version. Snapshot of the Site A instance taken before starting.

Prior labs: holodeck91-03

Required skills:

  • Holodeck 9.1 lifecycle cmdlets (New-HoloDeckConfig, Import-HoloDeckConfig, New-HoloDeckNetworkConfig, Set-HoloRouter, New-HoloDeckInstance and its built-in precheck phase) from holodeck91-01/02
  • IP address planning and CIDR notation
  • VCF management domain architecture (SDDC Manager, vCenter, NSX) — note the 9.1 unified management services runtime
  • BGP routing concepts or comfort with static route inspection (vtysh)
  • Disaster recovery concepts: RTO/RPO, failure domains, active-passive vs stretched topologies
📸 Starting State: S3-91 — Management Domain Deployed (VCF 9.1.0.0)

Lab Environment

Single physical ESXi host (or modest vSphere cluster) running two Holodeck 9.1 instances. Site A (production role) uses the 10.1.0.0/20 management supernet; Site B (DR role) uses 10.2.0.0/20. Holodeck 9.1 creates both site config files from one New-HoloDeckConfig call, but each site's networking is still generated with New-HoloDeckNetworkConfig -Site a / -Site b (default CIDRs 10.1.0.0/20 and 10.2.0.0/20, or custom /20 CIDRs with -VLANRangeStart) and the router is switched to dual-site with Set-HoloRouter -dualsite before either site is deployed. The HoloRouter provides routing between sites plus the 9.1 built-in service stack: Technitium DNS (port 5380), HashiCorp Vault (8200), Authentik SSO (9443), and an authenticated Webtop (30000), all behind an HTTPS reverse proxy. Nested components are named by FQDN (sddcmanager-a, vc-mgmt-a, nsx-mgmt-a under site-a.vcf.lab; -b under site-b.vcf.lab); all addressing is carved from each site's /20, so read the actual IPs with Get-HoloDeckAppNetwork -Site a|b.

Important

Holodeck 9.1 session-context rule: after New-HoloDeckConfig, the SITE A configuration is loaded into your PowerShell session automatically. The Site B configuration exists on disk but is NOT loaded. Every Get-HoloDeck* query and every deployment action operates against the currently loaded site context. Use Import-HoloDeckConfig -ConfigId <id> -Site b to switch context before any Site B operation. Forgetting this is the number-one new operator error in 9.1 dual-site work.

Mermaid source
graph TB
  PHY[Physical ESX 8.0 U3 / 9.0 host - 1536 GB RAM, 96 CPU, 5 TB for dual-site VCF 9.1] --> HR[HoloRouter 9.1<br/>HTTPS reverse proxy<br/>Technitium DNS :5380 / Vault :8200 / Authentik :9443]
  HR -->|Per-site network config + Set-HoloRouter -dualsite| CFG[New-HoloDeckConfig creates Site A + Site B config files<br/>New-HoloDeckNetworkConfig -Site a / -Site b]
  subgraph SiteA["Site A - Production (10.1.0.0/20)"]
    ESXA[4x Nested ESX 9.1 esx-01a..04a.site-a.vcf.lab]
    SDDCA[sddcmanager-a.site-a.vcf.lab]
    VCA[vc-mgmt-a.site-a.vcf.lab]
    NSXA[nsx-mgmt-a.site-a.vcf.lab]
  end
  subgraph SiteB["Site B - DR (10.2.0.0/20)"]
    ESXB[4x Nested ESX 9.1 esx-01b..04b.site-b.vcf.lab]
    SDDCB[sddcmanager-b.site-b.vcf.lab]
    VCB[vc-mgmt-b.site-b.vcf.lab]
    NSXB[nsx-mgmt-b.site-b.vcf.lab]
  end
  HR --> SiteA
  HR --> SiteB
  SiteA <-->|Cross-site routing via HoloRouter| SiteB
  IPS[Component IPs: Get-HoloDeckAppNetwork -Site a / -Site b] -.-> HR

IP Addressing

NetworkPurposeVLAN
10.1.0.0/20Site A management supernet (auto-generated by New-HoloDeckConfig)Site A VLAN block (auto-assigned — record actual IDs from generated config)
10.1.1.0/24Site A nested ESX host managementSite A management VLAN
10.2.0.0/20Site B management supernet (auto-generated in the same pass)Site B VLAN block — default 40-58 with 10.2.0.0/20 (Site A default 10-25 with 10.1.0.0/20); custom ranges via -VLANRangeStart must not overlap Site A
10.2.1.0/24Site B nested ESX host managementSite B management VLAN

Credentials

SystemUsernamePassword
HoloRouter SSHrootSet at HoloRouter OVA deployment (9.1 default documented in toolkit docs)
SDDC Manager A / Badministrator@vsphere.localSet in the generated bring-up spec. API calls use the doubled master password convention (e.g. VMware123!VMware123!) — verify this convention still holds on your 9.1 build
vCenter A / Badministrator@vsphere.localSame SSO password as the site's SDDC Manager
NSX Manager A / BadminSet in generated bring-up spec; doubled password for API auth
HoloRouter web services (Vault/Authentik/Technitium/Webtop)admin (service-specific)Each HoloRouter service keeps its own documented login behind the HTTPS reverse proxy: Vault root token, Technitium admin, Webtop admin and GitLab root (defaults in the Holodeck 9.1 docs, Enhanced Services; see holodeck91-04). Authentik (akadmin per Getting Started, admin per the release notes; verify) is the identity provider that Initialize-Authentik and Set-VCFSSOConfiguration wire into VCF SSO, not a gate in front of these services.

Tasks

Task 1 Generate the dual-site configuration in a single pass

manageability

This is the headline 9.1 workflow change (Holodeck 9.1 GA announcement): one New-HoloDeckConfig call now creates the Site A and Site B config files together so parallel site deployments cannot race each other (9.1 GA blog). The per-site network allocation is still an operator step — New-HoloDeckNetworkConfig -Site a / -Site b with non-overlapping /20 CIDRs and VLAN ranges, then Set-HoloRouter -dualsite — and the community KB documents real failures from overlapping ranges. A VCDX panelist will recognise the 'automate the race-prone step, keep the design decision explicit' pattern.

Step 1

SSH to the HoloRouter (ssh root@<holorouter-ip>) and start PowerShell with pwsh; the Holodeck 9.1 cmdlets and PowerCLI are preinstalled on the HoloRouter (Holodeck 9.1 docs). Record the HoloRouter OVA build you deployed as the toolkit version for this lab.

pwsh session open on the HoloRouter with the Holodeck cmdlets available (for example Get-Command New-HoloDeckConfig returns the cmdlet). If the HoloRouter is a 9.0.x OVA, stop and deploy the 9.1 HoloRouter OVA first.
Do not mix a 9.0.x HoloRouter with this 9.1 lab. Cmdlet names overlap but parameter sets differ (notably the mandatory -Site on Import-HoloDeckConfig).
Step 2

Create the configuration. Run New-HoloDeckConfig with your target host parameters (-TargetHost, -UserName, -Password, -Description). In 9.1 this single invocation creates TWO configuration files: one for Site A and one for Site B. Then generate the per-site networking: New-HoloDeckNetworkConfig -Site a -MasterCIDR 10.1.0.0/20 and New-HoloDeckNetworkConfig -Site b -MasterCIDR 10.2.0.0/20 (custom: /20 CIDRs plus -VLANRangeStart per site), and switch the router to dual-site with Set-HoloRouter -dualsite.

Two configurations recorded (Site A loaded into the current session, Site B on disk); Get-HoloDeckSubnet -Site a and -Site b show non-overlapping subnets and VLANs; Set-HoloRouter -dualsite completes with one VRF per site. Record the ConfigId: the same ID is loaded with -Site a or -Site b.
The 9.1 release notes say the two-file generation exists to eliminate race conditions when the two sites are deployed in parallel. Non-overlap of CIDRs and VLANs still depends on the per-site New-HoloDeckNetworkConfig inputs (defaults, or the -MasterCIDR / -VLANRangeStart you choose).
Version Notes: 9.0.x behaviour for comparison: New-HoloDeckConfig created one config and the per-site network configs came from the same New-HoloDeckNetworkConfig -Site a / -Site b runs — that step is unchanged in 9.1; what 9.1 adds is the second config file and the -Site parameter on Import-HoloDeckConfig (KB thread 10 covers the overlap mistakes).
Step 3

Verify which site context is loaded: inspect $config (the Holodeck docs' method for checking the loaded configuration) and confirm it holds the Site A configuration (Site A description and 10.1.x addressing). Get-HoloDeckConfig only lists the available configurations; it does not show which one is loaded.

$config shows the Site A configuration. Site B's configuration exists on disk but is not loaded in this session.
Step 4

Inspect both generated configuration files on disk (location per your 9.1 install, typically under the holodeck runtime config directory — verify the exact path against your live 9.1 toolkit). Compare side by side: management CIDR (expect 10.1.0.0/20 vs 10.2.0.0/20 or your chosen supernets), VLAN ranges, port-group names, and reserved component IPs for SDDC Manager, vCenter, and NSX on each site.

Two config files with symmetric structure and strictly non-overlapping CIDRs, VLAN IDs, and port-group names. Component IP plans mirror each other across the 10.1.x / 10.2.x split.
Diff the two files programmatically (Compare-Object on parsed JSON) and save the diff as a design artifact — it is a compact, defensible record of exactly what differs between your two sites: addresses only, not architecture.
Step 5

Practice the site context switch. Load Site B: Import-HoloDeckConfig -ConfigId <id> -Site b (the same ConfigId, with -Site b). Inspect $config again and confirm it now holds the Site B configuration. Then switch back with Import-HoloDeckConfig -ConfigId <id> -Site a.

Active configuration context flips to Site B (10.2.x addressing visible in queries), then back to Site A. No errors on either switch.
Every subsequent cmdlet in this lab is site-context-sensitive. Adopt the habit now: before ANY deployment or query action, inspect $config and confirm which site you are pointed at. Running a destructive action against the wrong site context is the 9.1 equivalent of the classic 'wrong terminal window' incident.

Validation Gate

Check: Two configuration files exist (Site A + Site B) from a single New-HoloDeckConfig pass. Diff confirms non-overlapping CIDRs/VLANs/port groups. Import-HoloDeckConfig -Site b successfully switches session context and back.

Expected: Dual-site configuration generated and verified; operator can deliberately control site context.

Common Errors

Only one configuration file is generated
Cause: The HoloRouter is a 9.0.x OVA, or a 9.1 pre-GA build without the dual-generation behaviour
Fix: Check the HoloRouter OVA build you deployed; if it is not 9.1 GA or later, deploy the 9.1 HoloRouter OVA and re-run New-HoloDeckConfig from pwsh on that HoloRouter.
Get-HoloDeck* queries return Site A data even though you intended to work on Site B
Cause: Site B config was generated but never loaded — Site A is auto-loaded by default in 9.1
Fix: Run Import-HoloDeckConfig -ConfigId <id> -Site b, then re-run the query. Confirm with $config (Get-HoloDeckConfig only lists configurations) before proceeding.
New-HoloDeckConfig warns about template config integrity
Cause: The template config.json was hand-modified (behavior introduced in 9.0.1 template integrity validation, still present in 9.1)
Fix: Do not edit the template file. Restore the pristine template from the toolkit distribution and pass customizations via cmdlet parameters instead.
📋 KB: 9.0.1 GA announcement — template integrity validation

Task 2 Precheck both sites and verify the 9.1 dual-site fixes

availability

9.1 hardened exactly the failure points that broke 9.0.x dual-site deployments: VLAN range handling, Site B port groups and installer validation (Holodeck 9.1 GA announcement). Holodeck runs its prechecks at the start of New-HoloDeckInstance (Concepts > Built-In PreChecks) — there is no separate Start-HoloDeckPrecheck cmdlet in the 9.1 command reference — so the discipline is to treat that interactive precheck phase as the gate for each site before hours of deployment time are committed.

Step 1

With the Site A context loaded, start New-HoloDeckInstance -Site a [options] and review the precheck phase before it proceeds: binary/content validation, target host reachability, trunk port group and datastore selection, NTP, and the 9.1 resource soft-validation. If Site A is already deployed (holodeck91-03), skip this step and record its deployment evidence instead.

All Site A prechecks pass (or Site A already healthy); precheck output captured as evidence.
Step 2

Open a second PowerShell tab, load Site B (Import-HoloDeckConfig -ConfigId <id> -Site b) and start New-HoloDeckInstance -Site b [options]; treat the precheck phase as the gate — review every check and abort (Ctrl+C) before deployment begins if any fails. Record the output.

All Site B prechecks pass, including the Site B port-group checks that were failure-prone on 9.0.x (distinct port group per site, introduced in 9.0.1 and hardened in 9.1). Deployment may continue into Task 3 or be aborted for remediation.
Step 3

Verify storage and memory headroom for the second site on the physical host BEFORE deploying. From the physical ESXi host client (or esxtop), record current consumed memory and datastore free space. Site B needs approximately the same footprint as Site A — the Holodeck 9.1 docs size single-site VCF 9.1 at 650 GB RAM / 2.2 TB and dual-site at 1536 GB / 5 TB, so plan roughly 650 GB RAM and 2.2-2.8 TB of additional disk, verified against your actual Site A consumption.

Documented headroom calculation: physical capacity minus Site A actuals leaves enough for Site B plus a safety buffer (target: keep projected total below ~90% of physical RAM).
The dominant dual-site failure mode is not networking — it is memory exhaustion mid-deployment of Site B, which destabilizes the already-running Site A. KB thread 19 (Site B vCenter not deploying) is this exact failure. If headroom is marginal, reduce Site B nested host sizing before deploying, not after it fails.
Step 4

Verify the staged binaries for Site B: the physical target host must be ESX 8.0 U3 or 9.0 (Holodeck minimum), and the ESX ISO and VCF Installer staged under /holodeck-runtime/bin must be the VCF 9.1.0.0 BOM versions (ESX 9.1) — the nested Site B hosts are built from that ISO.

Staged ESX 9.1 ISO and VCF Installer match the 9.1.0.0 BOM; physical host on ESX 8.0 U3 or 9.0.

Validation Gate

Check: Precheck phase of New-HoloDeckInstance passes for both -Site a and -Site b (output captured). Headroom calculation documented. Staged ESX 9.1 ISO / VCF Installer confirmed; physical host ESX 8.0 U3 or 9.0.

Expected: Both sites cleared for deployment with documented capacity evidence.

Common Errors

Site B precheck fails on port group validation
Cause: Manually created or leftover port groups from a previous 9.0.x dual-site attempt collide with the auto-generated Site B port-group plan
Fix: Remove stale Holodeck port groups from the physical host (verify nothing is attached first), then re-run the precheck. Do not hand-create Site B port groups on 9.1 — the toolkit owns them.
📋 KB: Holodeck 9.1 GA announcement: Site B port group issues resolved in 9.1
Precheck passes but you doubt the VLAN allocation
Cause: Legacy habit from 9.0.x where VLAN range handling had defects
Fix: 9.1 fixed VLAN range handling (Holodeck 9.1 GA announcement), but trust-and-verify: dump both site configs' VLAN assignments and confirm zero intersection. File a toolkit issue if you find overlap — that would be a regression.

Task 3 Deploy the Site B management domain alongside Site A

availability

Deploying a second full VCF instance next to a live one is a controlled-risk exercise in shared-infrastructure thinking: the two sites are logically independent (separate SDDC Manager, vCenter, NSX, vSAN) but physically entangled (same host, same HoloRouter). VCDX panels probe exactly this distinction — 'what is actually isolated here, and what only looks isolated?'

Step 1

Take a snapshot of all Site A VMs before starting (Get-VM pattern from earlier labs). Label it clearly (e.g. 'pre-siteb-deploy').

Snapshot exists for every Site A Holodeck VM. This is your rollback line if Site B deployment destabilizes the host.
Step 2

Confirm the Site B context is loaded ($config holds the Site B configuration), then start (or continue) the deployment: New-HoloDeckInstance -Site b [options] from the second PowerShell tab (Start-HoloDeckInstance is for powering on an existing instance, not for deployment). Use -Verbose for a progress stream. Alternative path: 9.1 also supports driving this deployment from the GitOps/UI pipeline (Holodeck 9.1 GA announcement) — this lab uses the cmdlet path for repeatability; note the alternative in your runbook.

Site B deployment proceeds through nested ESX provisioning, installer validation (hardened in 9.1), management appliance deployment, and VCF bring-up. Expect 90-150 minutes; longer under resource contention with Site A.
Monitor physical host memory every 10-15 minutes during deployment (esxtop). If utilization exceeds ~95%, Site B components will start failing in confusing ways (vCenter services crashing, installer timeouts) and Site A becomes collateral damage.
Step 3

While deployment runs, verify Site A remains healthy: SDDC Manager-A UI reachable, vCenter-A shows all hosts connected, vSAN health green. Record any degradation with timestamps.

Site A stays fully operational throughout Site B deployment. Any Site A degradation correlates with physical resource pressure and should be documented as a shared-infrastructure risk observation for the design exercise.
Step 4

When deployment completes, query both instances. Switch contexts as needed and run Get-HoloDeckInstance for each site's InstanceID.

Two instances reported: Site A and Site B, both Running, both VCF 9.1.0.0.
Step 5

Log in to the Site B management plane: SDDC Manager-B (sddcmanager-b.site-b.vcf.lab), vCenter-B (vc-mgmt-b.site-b.vcf.lab), NSX Manager-B (nsx-mgmt-b.site-b.vcf.lab); IPs from Get-HoloDeckAppNetwork -Site b. Verify the management domain shows Active with all hosts commissioned. Note: 9.1's unified management services runtime means the SDDC Manager / VCF Operations footprint differs from what the 9.0 labs describe — record what you actually observe.

Site B management domain Active: hosts connected, vSAN green, NSX transport nodes up. Addresses match the auto-generated Site B config exactly.

Validation Gate

Check: Get-HoloDeckInstance shows Site A and Site B both Running on VCF 9.1.0.0. Site B SDDC Manager/vCenter/NSX all healthy. Site A suffered no persistent degradation.

Expected: Two independent VCF 9.1 management domains operational on the dual-site topology.

Common Errors

Site B deployment actions execute against Site A objects
Cause: Session context was still Site A when deployment cmdlets ran
Fix: Stop immediately. Verify damage scope on Site A (snapshot from step 1 is your recovery line). Re-import the Site B config with -Site b and re-run. Add a context-check step to your personal runbook.
Site B vCenter deployment fails; vmware-vpostgres or other services crash on start
Cause: Physical host memory oversubscription, or Site B DNS not resolving during vCenter firstboot
Fix: Check physical host memory (esxtop); reduce Site B nested host memory or count if >95%. Verify Site B DNS against the HoloRouter's Technitium DNS service — confirm Site B zones exist (Task 4).
📋 KB: Thread 19: Holodeck dual-site — Site B vCenter not deploying
Installer validation rejects the Site B environment
Cause: 9.1's hardened VCF installer validation catching a real problem earlier than 9.0.x would have (wrong ESX ISO/VCF Installer build staged for the version, storage shortfall, network mismatch)
Fix: Read the validation failure precisely — 9.1 validation failing fast is a feature. Fix the flagged precondition rather than looking for a bypass.

Task 4 Validate the dual-site topology — cross-site routing, DNS, and management-plane independence

manageability

A dual-site lab is only useful for DR rehearsal if the sites can actually reach each other AND fail independently. This task validates both properties. The 9.1 twist: DNS is now served by Technitium on the HoloRouter behind the HTTPS reverse proxy (Holodeck 9.1 docs), which changes how you inspect and extend cross-site name resolution compared to the 9.0.x dnsmasq procedures.

Step 1

Verify cross-site IP reachability. From a Site A source (Webtop or the HoloRouter shell), ping Site B SDDC Manager and vCenter (sddcmanager-b.site-b.vcf.lab and vc-mgmt-b.site-b.vcf.lab; IPs from Get-HoloDeckAppNetwork -Site b). Repeat in the reverse direction from Site B.

Bidirectional replies, <2ms latency (same physical host). Both site supernets are routed via the HoloRouter.
Step 2

Inspect cross-site routing on the HoloRouter. SSH to the HoloRouter and review the routing state for both site supernets (vtysh 'show ip route' if FRR-based routing is in use on your build, or the routing method your 9.1 router uses — verify against live 9.1 toolkit, as the 9.1 router service stack changed substantially from 9.0.x).

Routes for both 10.1.0.0/20 and 10.2.0.0/20 present and pointing at the correct interfaces.
Step 3

Validate cross-site DNS with Technitium. Open the Technitium DNS console on the HoloRouter management IP, port 5380 (behind the HTTPS reverse proxy — authenticate as required on your build). Verify both Site A and Site B zones/records were provisioned by the dual-site generation. Test resolution both ways: from Site A, resolve a Site B FQDN; from Site B, resolve a Site A FQDN.

Technitium shows record sets for both sites. nslookup of cross-site FQDNs resolves correctly from both directions.
Version Notes: 9.0.x labs (holodeck-06 Task 3) had you hand-edit dnsmasq configs for cross-site resolution. On 9.1, Technitium replaces dnsmasq (open at release; settled 24 Sep 2026 — docs/index.md Getting Started and maintainer Discussion #178, 15 Sep 2026), so the old dnsmasq fix does not apply; residual: confirm no dnsmasq pod on the live router (kubectl get pods -A) before applying any legacy DNS workaround.
Step 4

Verify management-plane independence. Confirm that Site A and Site B each have their own SDDC Manager, vCenter, SSO domain, NSX management cluster, and vSAN datastore — i.e., no shared management state between sites other than the HoloRouter transit/DNS layer.

Component inventory table for both sites showing fully parallel, independent management stacks.
Step 5

Perform a controlled independence test: restart a non-critical Site B management service (or briefly disconnect a Site B nested host NIC), and confirm Site A registers zero impact. Restore immediately.

Site A health remains green throughout the Site B disturbance. Document this as evidence of logical failure-domain separation — while noting the shared physical host remains a common failure domain.

Validation Gate

Check: Bidirectional cross-site ping and DNS resolution pass. Both site supernets routed. Component inventory confirms parallel independent management stacks. Site B disturbance produced no Site A impact.

Expected: Dual-site topology validated: connected where intended, isolated where intended.

Common Errors

Cross-site DNS fails but cross-site ping works
Cause: Technitium zones for the second site missing or the client is pointed at a stale DNS address from a previous 9.0.x deployment
Fix: Check client resolv.conf/DNS settings point at the HoloRouter DNS IP. Inspect Technitium zone list on :5380. Re-sync the instance config if records are missing.
📋 KB: Thread 30: DNS resolution in Holodeck dual-site deployment (9.0.x-era; verify applicability on 9.1 Technitium stack)
Cannot reach Technitium/Vault/Authentik consoles on their documented ports
Cause: 9.1 places the HoloRouter services behind an HTTPS reverse proxy at vcf.lab FQDNs; direct-port habits from 9.0.x may not apply, and the console must use the HoloRouter as its DNS server for FQDN access
Fix: Use the service's https://<service>.vcf.lab FQDN (or the HoloRouter management IP with the documented port), accept the proxy certificate, and log in with the service's own documented credential (Vault root token, Technitium/Webtop admin, GitLab root). Authentik is the VCF SSO identity provider, not the login for these consoles.

Task 5 Design exercise — auto-generation, drift, and the dual-site vs stretched decision

recoverability

This is the defense-facing payload of the lab. You now have first-hand evidence of what two-file configuration generation buys you (no race conditions between parallel site builds, symmetric configs) and what still depends on your per-site network inputs and what it does not (shared physical failure domain, no automatic DR orchestration). Convert that evidence into VCDX-grade positions using RCAR.

Step 1

Write a one-page position: 'Why automated dual-site configuration generation reduces configuration drift.' Ground it in what you observed: (a) both sites derive from one allocation computation — no human transcription between sites; (b) the config diff you produced in Task 1 shows addresses differ, architecture does not; (c) the 9.0.x failure modes this eliminates (KB thread 10 CIDR overlap, Site B port group collisions) were all human-transcription errors; (d) race-condition elimination when generating/deploying sites in parallel.

Position paper connecting the tooling change to the drift/error classes it eliminates, with the Task 1 diff attached as evidence
Step 2

Define the production scenario your lab models: Site A primary, Site B DR, active-passive, with target RPO/RPO. State explicitly which lab properties transfer to production (symmetric config generation, independent management stacks, cross-site DNS) and which do not (shared physical host, <2ms inter-site latency, single HoloRouter as shared transit).

Scenario statement with an honest lab-vs-production applicability table
Step 3

Build the dual-site vs stretched-cluster comparison. For each topology document: failure domain boundaries, latency requirements (stretched vSAN needs ~<5ms RTT and L2 adjacency; dual-site async replication tolerates WAN latency), RPO achievable (near-zero sync vs minutes-hours async), operational complexity (witness placement and split-brain handling vs failover runbooks), and licensing/cost. Conclude with which business profiles each fits.

Comparison matrix plus a defensible recommendation logic: 'choose stretched when..., choose dual-site DR when...'
Step 4

Complete an RCAR worksheet for the dual-site design (minimum: 5 requirements, 4 constraints, 4 assumptions, 6 risks with impact and mitigation). Include at least one risk unique to 9.1 tooling: e.g., wrong-site-context operator error enabled by fast context switching.

Completed RCAR worksheet
Step 5

Prepare panel-response drills: answer aloud, in under two minutes each — (a) 'Your two sites were generated from one template. What happens to your symmetry claim after two years of independent Day-2 change?' (b) 'Why not stretch the cluster instead?' (c) 'Where exactly does your failure-domain boundary sit in this lab, and where would it sit in production?'

Recorded or written answers demonstrating command of drift-over-time, topology selection, and failure-domain reasoning

Validation Gate

Check: All five artifacts exist: drift position paper with diff evidence, scenario + applicability table, dual-site vs stretched matrix, RCAR worksheet, panel drill answers.

Expected: Design documentation strong enough to survive follow-up probing in a VCDX defense.

Final Validation

A dual-site Holodeck 9.1 deployment is operational: Site A and Site B config files generated together (no parallel-deployment race), per-site networking prepared with New-HoloDeckNetworkConfig and Set-HoloRouter -dualsite with non-overlapping inputs, Site B deployed alongside a live Site A, cross-site routing and Technitium DNS validated bidirectionally, and management-plane independence demonstrated. The design exercise converts the tooling change into defense material: automation of the cross-site consistency problem, and a reasoned dual-site vs stretched topology position.

✓ Task 1: Single New-HoloDeckConfig pass produced Site A + Site B configs; context switching via Import-HoloDeckConfig -Site b verified → Two config files, non-overlapping allocation confirmed by diff, session context controllable

✓ Task 2: Prechecks pass for both sites with documented capacity headroom → New-HoloDeckInstance precheck phase green for -Site a and -Site b; staged ESX 9.1 binaries confirmed; physical host ESX 8.0 U3 or 9.0

✓ Task 3: Site B deployed alongside live Site A → Get-HoloDeckInstance shows both instances Running on VCF 9.1.0.0; Site A undisturbed

✓ Task 4: Cross-site reachability, DNS, and independence validated → Bidirectional ping + Technitium DNS resolution; independent management stacks; Site B disturbance invisible to Site A

✓ Task 5: Design exercise complete → Drift position, topology comparison, RCAR, and panel drills documented

Cleanup / Restore

Snapshot: holodeck91-05-complete

• Snapshot all VMs of BOTH sites with a clear label (e.g. 'holodeck91-05-complete') — subsequent Day-2 labs (holodeck91-08/09) can run against either site

• Archive the Task 1 config diff, RCAR worksheet, and topology comparison to your VCDX evidence folder

• If physical resources are constrained, Stop-HoloDeckInstance the Site B instance (do NOT remove it) to reclaim RAM for later labs — record which site is stopped in your lab notebook

• If you must fully reclaim Site B: use Remove-HoloDeckInstance against the SITE B context only, and verify the site context twice before executing. Do not use -ResetHoloRouter on a shared or dual-site router — it tears down global routing/DNS state for everything on the host

Design Reflection (VCDX)

Two defense threads converge in this lab. THREAD 1 — configuration drift as an availability risk: panels increasingly probe operational consistency, not just steady-state architecture. The 9.1 two-file generation is a concrete instance of an anti-drift pattern: generate related configurations together rather than hand-building each, while the per-site network inputs remain a deliberate design choice you must be able to explain.

Be ready to generalize: the same argument justifies infrastructure-as-code for production VCF (and the toolkit's own GitOps path), golden-image host provisioning, and declarative NSX policy. Also be ready for the counterpunch: generation guarantees symmetry at time zero only — without ongoing enforcement (drift detection, redeploy-from-source discipline), two years of independent Day-2 change erodes the symmetry claim. THREAD 2 — topology selection: dual-site active-passive DR vs stretched cluster is a classic panel fork.

Dual-site (this lab): independent failure domains, tolerant of WAN latency, asynchronous replication RPO in minutes, explicit failover with runbooks — simpler to reason about, slower to recover. Stretched: near-zero RPO and transparent HA restart across sites, but demands <5ms RTT, L2 adjacency or overlay stretch, witness placement, and rigorous split-brain design — a harder operational commitment. The VCDX move is to bind the choice to stated business requirements (RPO/RTO, distance between sites, network budget, operational maturity) rather than to technology preference.

Finally, name the lab's honesty caveat before the panel does: both 'sites' share one physical host and one HoloRouter, so the lab rehearses logical topology and operational workflow, not genuine site-level failure isolation.

Requirements

  • Two complete, independently manageable VCF 9.1 management domains (Site A production role, Site B DR role)
  • Non-overlapping network allocation across sites, from documented per-site inputs (default /20s and VLAN blocks, or explicit -MasterCIDR / -VLANRangeStart) recorded in the design
  • Bidirectional cross-site management-plane reachability (IP and DNS) for replication and failover tooling
  • Site B deployable next to a live Site A without disrupting Site A operations
  • Operator ability to target either site unambiguously (session context control) for all lifecycle actions
  • Design documentation sufficient to defend the dual-site vs stretched decision against stated RPO/RTO targets

Constraints

  • Single physical host: both sites share one hardware failure domain and one HoloRouter transit/DNS layer — site-level disaster isolation cannot be genuinely rehearsed
  • Physical capacity: dual-site VCF 9.1 is documented (recommended) at 96 CPU / 1536 GB RAM / 5 TB SSD-NVMe (768 GB / 64 CPU / 2.5 TB was the VCF 9.0 dual-site row); Site B is not achievable on a 1 TB host without accepting the soft-validation warning and degraded performance
  • Holodeck 9.1 requires the physical target host on ESX 8.0 U3 or 9.0; nested hosts are built from the staged ESX ISO of the VCF release, so the 9.1.0.0 BOM binaries must be staged before Site B
  • Lab latency (<2ms) is unrepresentative of production inter-site links; latency-sensitive conclusions (stretched vSAN feasibility) cannot be validated here
  • 9.1 HoloRouter service stack (Technitium/Vault/Authentik behind HTTPS proxy) invalidates 9.0.x DNS/service procedures; exact ingress paths and defaults must be verified per build

Assumptions

  • Site A instance (holodeck91-03) is healthy and snapshotted before Site B deployment begins
  • The auto-generated Site B allocation does not collide with any OTHER instance on a shared physical host (coordinate InstanceIDs and CIDRs with colleagues on shared hardware)
  • The doubled-password API convention from 9.0.x persists on 9.1 builds (verify on first API call)
  • Cmdlet-driven deployment path chosen for repeatability; GitOps/UI path (Holodeck 9.1 GA announcement) assumed equivalent in outcome but not exercised here
  • Production mapping assumes real dual-site would add: independent physical infrastructure per site, WAN latency, replication tooling, and failover orchestration not present in the lab

Risks

  • Physical memory exhaustion during Site B deployment destabilizes live Site A — IMPACT: both sites degraded, possible total lab loss. MITIGATION: pre-deployment headroom calculation (Task 2), esxtop monitoring during deploy, Site A snapshots as rollback line, reduce Site B sizing when marginal.
  • Wrong-site-context operator error: destructive cmdlet executed against the unintended site — IMPACT: damage to the healthy site. MITIGATION: $config context check before every action; personal runbook rule; snapshots before any destructive step.
  • Stale 9.0.x artifacts (port groups, DNS records, module versions) contaminate the 9.1 dual-site deployment — IMPACT: precheck failures or subtle routing/DNS misbehavior. MITIGATION: clean the host of prior-version artifacts; never mix toolkit module versions in one session.
  • Symmetry erosion over time: Day-2 changes applied to one site only — IMPACT: DR site no longer representative; failover fails when needed. MITIGATION: change-management rule that Day-2 changes apply to both sites or are logged as accepted asymmetry; periodic config re-diff against the generated baseline.
  • Shared HoloRouter as hidden single point of failure for BOTH sites' transit and DNS — IMPACT: one router fault severs cross-site operations and name resolution everywhere. MITIGATION: acknowledge explicitly in the design; in production map this to redundant WAN/DNS infrastructure; see holodeck91-10 for the distributed VNA alternative to single-router networking.
  • Treating lab DR rehearsal as production DR validation — IMPACT: false confidence in RPO/RTO claims. MITIGATION: applicability table (Task 5 step 2) states exactly which conclusions transfer; production targets validated only by production-representative testing.

Self-Assessment Discussion Prompts

  1. Explain what the 9.1 two-file New-HoloDeckConfig removes (parallel-deployment races) and what it does not remove (overlapping CIDR or VLAN inputs to New-HoloDeckNetworkConfig, shared physical failure domain).
  2. Your two sites are symmetric today because one tool generated both. Describe a governance mechanism that keeps them defensibly symmetric after two years of Day-2 operations.
  3. The 9.1 release notes say dual generation 'eliminates race conditions when deploying dual-site in parallel.' What was the race, and what does its existence tell you about the 9.0.x allocation model?
  4. Defend choosing dual-site active-passive DR over a stretched cluster for a business with RPO 1 hour / RTO 4 hours and 40ms inter-site RTT. Now invert it: what requirement changes would flip your recommendation to stretched?
  5. In this lab, what is the true failure-domain boundary between Site A and Site B? List every shared component and classify each as acceptable-in-lab-only or acceptable-in-production.
  6. A panelist asks: 'Walk me through why Site B got the VLANs it got.' Answer from your New-HoloDeckNetworkConfig -Site b inputs (default 10.2.0.0/20 with VLANs 40-58, or the -VLANRangeStart you set) and Get-HoloDeckSubnet output.
  7. How would you extend this dual-site foundation toward a rehearsable failover: what replication, DNS failover, and runbook elements are missing between this lab and a testable DR capability?

Extensions

Drive the Site B Deployment via the GitOps Pipeline

Repeat the Site B deployment using the 9.1 GitOps path (HoloRouter OVA with GitOps enabled, GitLab-driven pipeline, Holodeck 9.1 GA announcement) instead of cmdlets. Compare: auditability of the pipeline vs verbose cmdlet logs, rollback story, and how the GitOps path handles the dual-site configuration pair. Document which path you would standardize on for a team lab and why.

same

Dual-Site Workload Domain Rehearsal

With both management domains up, deploy a small VI workload domain on each site (see holodeck-05 methodology, adapted to 9.1) and document the full dual-site estate: per-site licensing, capacity, and the management-plane inventory a DR runbook would need. Note where 9.1's unified management services runtime changes the component inventory vs the 9.0 twin lab.

harder

Cross-Site Failover Runbook and Tabletop Exercise

Author a complete Site A to Site B failover runbook (detection, decision tree, DNS cutover via Technitium, service verification, RTO accounting) and run it as a tabletop exercise without executing destructive steps. Time each phase against a 2-hour RTO target and identify the longest-lead-time step.

harder

Drift Injection and Detection

Deliberately introduce three configuration drifts on Site B (e.g., an extra port group, a changed NTP source, a modified DNS record) and build a detection script that diffs live state against the generated configuration baseline. This turns the drift position paper from Task 5 into a working control.

same

⚠ Known Pitfalls (from Community KB)

Holodeck 9.0.2 dual-site IP config SUPERSEDED_IN_9_1
Problem: 9.0.x-era failure: Site A and Site B manually configured with overlapping CIDR ranges, causing routing conflicts after both sites deployed; HoloRouter reboot was part of the resolution.
Resolution: 9.1's two-file generation removes the parallel-deployment race; overlapping per-site network inputs are still possible and must be checked with Get-HoloDeckSubnet -Site a / -Site b. Remains relevant as the motivating failure story for Task 5's drift discussion, and the HoloRouter-state lesson still applies.
Holodeck 9.0.2 Dual Site B vCenter not deploying STILL_RELEVANT
Problem: Site B vCenter deployment failed due to insufficient remaining memory on the shared physical host after Site A allocation.
Resolution: Capacity, not tooling — fully applicable on 9.1. Mitigate with the Task 2 headroom calculation and live esxtop monitoring during Site B deployment; reduce Site B sizing when marginal.
Issue with DNS resolution in Holodeck dual-site deployment VERIFY_ON_9_1
Problem: Cross-site FQDN resolution failures in 9.0.x dual-site setups, resolved via dnsmasq forwarder configuration.
Resolution: 9.1 moves DNS to Technitium behind the HTTPS reverse proxy — the dnsmasq fix likely does not apply. Re-diagnose via the Technitium console (:5380) per Task 4; do not blindly apply the 9.0.x workaround.
Holodeck vcf 9 - Cannot select datastore STILL_RELEVANT
Problem: Deployment fails at storage selection due to full or misnamed datastore — commonly hit when deploying a second site into space already consumed by the first.
Resolution: Verify the documented Site B disk headroom (about 2.2 TB or more free) before deployment and that the datastore name in the generated config matches physical inventory. Clean orphaned VMs/snapshots from failed prior attempts.

References

  • Announcing the General Availability of Holodeck 9.1 — VMware Cloud Foundation BlogTier 1 — Official
    Official GA announcement: VCF 9.1.0.0/VVF 9.1.0.0/5.2.4 support, dual-site auto-generation, VNA cluster support, Day-2 operations, HoloRouter service stack.
  • Holodeck 9.1 DocumentationTier 1 — Official
    Official 9.1 docs — authoritative for New-HoloDeckConfig dual-generation behavior, Import-HoloDeckConfig -Site parameter, and precheck workflow. Verify all cmdlet parameters here before deployment.
  • Simplify Multisite PoCs Using Holodeck — VMware Cloud Foundation BlogTier 1 — Official
    Multisite deployment patterns with Holodeck; written against 9.0.2 but the topology reasoning carries into 9.1.
  • Holodeck 9.1 Command ReferenceTier 1 — Official
    Official 9.1 cmdlet reference. The 9.1 release notes add that New-HoloDeckConfig now generates Site A and Site B configs (Site A auto-loaded) and that Import-HoloDeckConfig takes a mandatory -Site parameter; the live cmd reference syntax for Import-HoloDeckConfig showed only -ConfigID when checked on 9 Oct 2026, so confirm parameters with Get-Help on the HoloRouter before deployment.
  • VCF Holodeck Toolkit — Broadcom Community ForumTier 1 — Official
    Threads 10 (dual-site IP config), 19 (Site B vCenter not deploying), 30 (dual-site DNS) document the 9.0.x failure modes this lab's 9.1 workflow addresses. Note corpus is 9.0.x-era — verify applicability on 9.1.
  • Holodeck 9.1 Release NotesTier 1 — Official
    Official 9.1 release notes: dual-site config auto-generation and the bug fixes this lab relies on (VLAN range start handling, Site B deployment issues including depot alias and search path, VCF Installer validation).
Was this page useful?
Type to search. ↑ ↓ to move, Enter to open, Esc to close.