Dual-Site Holodeck Deployment
Objectives
- Plan the dual-site topology with the documented defaults (Site A 10.1.0.0/20, VLANs 0 and 10-25; Site B 10.2.0.0/20, VLANs 40-58) or custom /20 CIDRs and VLAN starts, and size the host from the dual-site requirement row
- Execute the documented dual-site sequence: New-HoloDeckNetworkConfig -Site a / -Site b, Set-HoloRouter -dualsite, New-HoloDeckInstance -Site a, Import-HoloDeckConfig and New-HoloDeckInstance -Site b in a second pwsh session
- Validate inter-site routing on the HoloRouter (L3 routing between the sites' VLANs, separate VRF tables) and bidirectional reachability between the two management planes
- Validate both management planes through their documented endpoints (site-a and site-b FQDNs) and record the two independent VCF Instances
- Map the dual-site lab to production multi-site designs using correct vSAN stretched-cluster facts (L2 or L3 between data sites, witness at a third site, preferred-site behaviour) in RCAR form
Prerequisites
A HoloRouter with the Holodeck 9.0.2 module, binaries staged for 9.0.2.0, and either a clean state (preferred: no instance yet) or a Site A instance deployed with the default network configuration. Physical capacity per the dual-site row (VCF 9.0 with Automation and vSAN OSA: 48 CPU, 768 GB RAM, 2.5 TB).
Prior labs: holodeck-02
Required skills:
- Holodeck configuration and deployment (holodeck-01, 02)
- IP subnetting and VLAN planning
- VCF multi-instance concepts (vcp-admin-10)
- vSAN stretched cluster and multi-site design fundamentals
Lab Environment
One HoloRouter serving two sites on one trunk port group: Site A (10.1.0.0/20, VLANs 0 and 10-25) and Site B (10.2.0.0/20, VLANs 40-58), each with its own nested management domain (4 hosts, VCF Installer, SDDC Manager, vCenter, NSX, VCF Operations). The HoloRouter routes between the sites' VLANs and keeps a separate VRF per site after Set-HoloRouter -dualsite.
Both sites run on one physical host behind one HoloRouter. They are two independent VCF Instances that share a router; they are not a stretched cluster and not a federated NSX. Design mapping in Task 5 must say so.
Mermaid source
graph LR
HR[HoloRouter: VRF-a + VRF-b, L3 routing between sites, DNS site-a.vcf.lab and site-b.vcf.lab]
subgraph A[Site A 10.1.0.0/20 VLANs 0,10-25]
A1[esx-01a..04a] --- AM[vc-mgmt-a, sddcmanager-a, nsx-mgmt-a, ops-a]
end
subgraph B[Site B 10.2.0.0/20 VLANs 40-58]
B1[esx-01b..04b] --- BM[vc-mgmt-a.site-b.vcf.lab, sddcmanager-a.site-b.vcf.lab, nsx-mgmt-a.site-b.vcf.lab, ops-a.site-b.vcf.lab]
end
HR --- A
HR --- BIP Addressing
| Network | Purpose | VLAN |
|---|---|---|
10.1.0.0/20 | Default Holodeck Site A master CIDR, subdivided by New-HoloDeckNetworkConfig into the Site A subnets (management, vMotion, vSAN, host overlay, edge uplinks, and others) | VLANs 0 and 10-25 (Site A default) |
10.2.0.0/20 | Default Holodeck Site B master CIDR (dual-site only) | VLANs 40-58 (Site B default) |
<external management network> | HoloRouter external management IP (DHCP or static, set at OVA deployment); the only external IP a Holodeck environment needs | External port group |
Credentials
| System | Username | Password |
|---|---|---|
| HoloRouter (SSH and pwsh) | root | Set in the OVA Additional settings at deployment |
| HoloRouter webtop | (no authentication in 9.0.2) | http://<holorouter-ip>:30000; the docs warn not to expose it externally |
| VCF Installer | admin@local | Holodeck documented default (Post Deployment table) |
| VCF Operations | admin | Holodeck documented default (Post Deployment table) |
| Management vCenter | administrator@vsphere.local | Holodeck documented default (Post Deployment table) |
| SDDC Manager | administrator@vsphere.local | Holodeck documented default (Post Deployment table) |
| Management NSX | admin | Holodeck documented default (Post Deployment table) |
| Nested ESX hosts | root | Holodeck documented default (Post Deployment table) |
Tasks
Task 1 Plan the dual-site topology and addressing
ManageabilityThe documentation fixes the shape: two /20 CIDRs, two VLAN ranges with minimum sizes, one HoloRouter with a VRF per site. Planning is choosing defaults or custom values and checking capacity.
Record the documented defaults: Site A 10.1.0.0/20 with VLANs 0 and 10 through 25; Site B 10.2.0.0/20 with VLANs 40 through 58. Custom values: each CIDR must be a /20; -VLANRangeStart is the first VLAN ID of the range (not an offset), with at least 16 consecutive valid IDs for Site A and 19 for Site B; -DNSDomain defaults to vcf.lab (FQDNs end in site-a.vcf.lab and site-b.vcf.lab).
Size from the dual-site column of the requirement tables: VCF 9.0 with Automation and ESA 64 CPU / 768 GB / 2.5 TB; with OSA 48 CPU / 768 GB / 2.5 TB; VVF 9.0 24 CPU / 512 GB / 2 TB; VCF 5.2 32 CPU / 1 TB / 4 TB; plus the HoloRouter. Decide management-only per site to fit.
Confirm the trunk port group allows both VLAN ranges (VLAN 4095 trunk does; on a cluster target the physical switches must carry 0, 10-25 and 40-58 or your custom ranges).
Decide the deployment order and sessions: the documentation runs New-HoloDeckInstance -Site a, then opens a new pwsh tab, imports the config and runs New-HoloDeckInstance -Site b. Plan two SSH sessions (or a terminal multiplexer) on the HoloRouter.
Draw the topology: one HoloRouter with VRF-a and VRF-b, two sites with their management planes and FQDN suffixes, the external workstation routes to both /20s.
Validation Gate
Check: CIDR/VLAN/DNS plan, capacity check, trunk coverage, session plan and diagram complete.
Expected: A dual-site plan consistent with the documented sequence.
Common Errors
Task 2 Execute the documented dual-site sequence
ManageabilityThe order matters: network configuration for both sites, then the HoloRouter in dual-site mode, then the two instances. Custom values must be repeated on every cmdlet or the defaults overwrite them.
SSH to the HoloRouter, run pwsh and create (or import) the configuration: New-HoloDeckConfig -Description 'dual-site' -TargetHost <target> -Username <u> -Password <p> (or Import-HoloDeckConfig -ConfigID <id>). One config per deployment still applies; the dual-site sequence is one deployment with two sites.
Generate the network configuration for both sites. Defaults: New-HoloDeckNetworkConfig -Site a -MasterCIDR 10.1.0.0/20 then New-HoloDeckNetworkConfig -Site b -MasterCIDR 10.2.0.0/20. Custom: add -VLANRangeStart <start> (and -DNSDomain) to each.
New-HoloDeckNetworkConfig -Site a -MasterCIDR 10.1.0.0/20
New-HoloDeckNetworkConfig -Site b -MasterCIDR 10.2.0.0/20Configure the HoloRouter for dual site: Set-HoloRouter -dualsite (separate VRF tables for each site, DNSMASQ and FRR configured). If you use custom CIDRs, also pass -CIDR with both ranges as the command reference shows.
Set-HoloRouter -dualsiteDeploy Site A: New-HoloDeckInstance -Version 9.0.2.0 -InstanceID holoa -Site a -ManagementOnly -vSANMode OSA -DepotType Online (custom networking: add -CIDR <site-a-cidr> -VLANRangeStart <site-a-start>). Answer the pre-check prompts as in holodeck-02.
New-HoloDeckInstance -Version 9.0.2.0 -InstanceID holoa -Site a -ManagementOnly -vSANMode OSA -DepotType OnlineOpen a second SSH session to the HoloRouter, run pwsh, Import-HoloDeckConfig -ConfigID <id>, and deploy Site B: New-HoloDeckInstance -Version 9.0.2.0 -InstanceID holob -Site b -ManagementOnly -vSANMode OSA -DepotType Online (custom networking: -CIDR <site-a-cidr>,<site-b-cidr> -VLANRangeStart <site-a-start>,<site-b-start>, both sites' values as the documentation shows).
Import-HoloDeckConfig -ConfigID <id>
New-HoloDeckInstance -Version 9.0.2.0 -InstanceID holob -Site b -ManagementOnly -vSANMode OSA -DepotType OnlineMonitor both runs (phases as in holodeck-02). The documented management-only time is 4-5 hours per site; running both concurrently loads the host, so expect longer.
If a site fails: re-run its exact command in its session (idempotent resume). If the networking must be rebuilt, Reset-HoloRouter followed by Set-HoloRouter -dualsite affects both sites: plan that.
Record both instances: Get-HoloDeckInstance -InstanceID holoa and -InstanceID holob.
From the outer layer, snapshot the HoloRouter and both instance VM sets (Stop-HoloDeckInstance each first) as holodeck-06-deployed.
Validation Gate
Check: Network configs for a and b, Set-HoloRouter -dualsite, both New-HoloDeckInstance runs completed in separate sessions, two instance records, snapshot.
Expected: Two nested management domains behind one HoloRouter.
Common Errors
Task 3 Validate inter-site routing and reachability
AvailabilityThe HoloRouter is the only path between the sites and the only path out. Prove routing in both directions before trusting any cross-site test.
On the HoloRouter compare the two sites: Get-HoloDeckSubnet -Site a and -Site b (VLAN sets 0,10-25 vs 40-58; subnets in 10.1.0.0/20 vs 10.2.0.0/20), Get-HoloDeckAppNetwork -Site a and -Site b, Get-HoloDeckBGPConfig -Site a and -Site b (different edge AS numbers per site).
From a Site A nested host console (or the webtop) ping the Site B management gateway and vc-mgmt-a.site-b.vcf.lab; from a Site B host do the reverse. The HoloRouter provides L3 routing between the VLANs of Site a and Site b (documentation, Holorouter Overview).
Resolve both sites' names from the HoloRouter: Get-HoloDeckDNSConfig -FQDN vc-mgmt-a.site-a.vcf.lab and -FQDN vc-mgmt-a.site-b.vcf.lab; from your workstation (holodeck-04 DNS) resolve both.
Add the Site B route on your workstation or site router (10.2.0.0/20 via the HoloRouter external IP) and open the Site B consoles by name.
Record the VRF behaviour you can observe: inter-site routing works, each site keeps its own address plan and DNS zone, and both share the HoloRouter's single external IP and firewall.
Measure cross-site latency between a Site A and a Site B nested host (same physical host, so sub-millisecond) and note why this is not representative of real inter-site links (Task 5).
Document the failure domain: one HoloRouter and one physical host serve both sites; a HoloRouter reboot (for example from Remove-HoloDeckInstance -ResetHoloRouter) takes both sites' routing and DNS down.
Validation Gate
Check: Network inventories compared; bidirectional reachability; DNS for both suffixes; workstation access to Site B; VRF observations; latency and shared-failure-domain notes.
Expected: Inter-site connectivity proven and its limits understood.
Common Errors
Task 4 Validate both management planes
AvailabilityTwo instances means two of everything. Validate each independently with the documented endpoints, then confirm they are independent VCF Instances (two SDDC Managers, two VCF Operations) rather than one fleet.
Site A: open https://vc-mgmt-a.site-a.vcf.lab, https://sddcmanager-a.site-a.vcf.lab, https://nsx-mgmt-a.site-a.vcf.lab and https://ops-a.site-a.vcf.lab; run the holodeck-02 Task 4 and 5 checks.
Site B: the documentation says to replace site-a with site-b in each FQDN: https://vc-mgmt-a.site-b.vcf.lab, https://sddcmanager-a.site-b.vcf.lab, https://nsx-mgmt-a.site-b.vcf.lab, https://ops-a.site-b.vcf.lab; repeat the checks.
Confirm independence: each site has its own SDDC Manager, vCenter, NSX Manager cluster and VCF Operations; neither knows about the other. Record that making them one fleet would mean integrating Site B into Site A's VCF Operations (vcp-admin-10) and that Holodeck does not do this for you.
Compare BOM versions in both VCF Operations inventories (same -Version, so identical) and note how two independently deployed instances can drift over time without fleet-level lifecycle.
Run Stop-HoloDeckInstance -InstanceID holob, verify Site A is unaffected (routing, DNS, consoles), then Start-HoloDeckInstance -InstanceID holob.
Repeat in the other direction for Site A.
Record the time to a healthy state for each site after Start-HoloDeckInstance.
Take the snapshot set holodeck-06-validated.
Validation Gate
Check: Both management planes validated; independence recorded; cross-site power-cycle tests passed; cold-start times recorded; snapshot taken.
Expected: Two healthy, independent VCF Instances on one Holodeck environment.
Common Errors
Task 5 Design exercise: map the dual-site lab to production multi-site architecture (RCAR)
AvailabilityThe lab gives you two instances behind one router. Production multi-site designs are different things: a vSAN stretched cluster within one instance, two instances in one fleet, or NSX Federation across instances. The exercise is to map correctly and to state the vSAN stretched-cluster facts the way the documentation does.
Write down what the lab is: two independent VCF Instances sharing one HoloRouter and one physical host, with inter-site L3 routing. Write down what it is not: not a stretched cluster, not NSX Federation, not a fleet (until integrated).
Stretched cluster facts from the VCF 9.0.1 documentation: vSAN stretched clusters support Layer 2 or Layer 3 between the two data sites (and L3 to the witness); the witness host resides at a third site; when the inter-site link fails, vSAN continues on the preferred site (the witness forms quorum with it) and vSphere HA restarts the secondary site's VMs there; objects become non-compliant but data stays available if the witness is lost. Write these as the requirements a stretched design must meet.
Multi-instance facts: two VCF Instances can be managed as one fleet by one VCF Operations (vcp-admin-10); consistent DFW across instances is NSX Federation (Global Manager, maximum 150 ms between locations); VCF Operations and VCF Automation connect to one identity broker (vcp-admin-04).
Build the RCAR for a two-site production design of your choice (active-active stretched management cluster, or two instances in one fleet with DR orchestration): Requirements (RPO/RTO, SSO, consistent security), Constraints (latency: stretched vSAN under 5 ms RTT between data sites, witness under 200 ms; Federation under 150 ms; bandwidth; the three-host minimum per site), Assumptions (third site for the witness, DNS/NTP per site, identity provider), Risks (inter-site link failure, witness loss, split management plane).
Write the design decision the lab cannot test: inter-site latency and bandwidth. Record the lab's sub-millisecond latency and state how you would test the real link (latency, loss, MTU) before committing a stretched design.
Write the failure scenarios for your design in a table (data-site failure, inter-site link failure, witness failure, management vCenter failure) with the documented behaviour and recovery for each, avoiding any 'read-only mode' or 'both sites keep writing' language.
Record how the lab helps and hinders: it exercises two management planes and inter-site routing; it cannot show stretched vSAN, Federation, real latency or independent power/cooling domains.
Prepare two defence answers: 'Why not stretch the management cluster across these two sites?' and 'What does one HoloRouter for two sites teach you about shared infrastructure services in production?'
Take the final snapshot set holodeck-06-complete and decide whether to keep both sites (resource cost) or remove Site B with Remove-HoloDeckInstance from its session.
Validation Gate
Check: Lab-versus-production statement; correct stretched-cluster and multi-instance facts; RCAR; untestable list; failure table; helps/hinders; two defence answers; snapshot.
Expected: A defensible mapping from the dual-site lab to production designs.
Common Errors
Final Validation
Two nested VCF 9.0.2 management domains deployed with the documented dual-site sequence behind one HoloRouter; inter-site routing and both management planes validated; the design exercise maps the lab correctly to production multi-site options with accurate vSAN stretched-cluster facts.
✓ Plan → Default or custom /20 CIDRs and VLAN starts (16/19 IDs), dual-site sizing, session plan
✓ Sequence → New-HoloDeckNetworkConfig a and b; Set-HoloRouter -dualsite; New-HoloDeckInstance -Site a and -Site b in separate sessions
✓ Routing → Bidirectional inter-site reachability; both DNS suffixes; workstation routes to both /20s
✓ Management planes → Both sites' consoles healthy; independence confirmed; power-cycle tests passed
✓ Records → Get-HoloDeckInstance for holoa and holob; cold-start times; snapshots
✓ Design exercise → Correct stretched-cluster facts; RCAR; failure table; untestable list; defence answers
✓ Failure domain → Shared HoloRouter and host recorded as the lab's single failure domain
✓ Decision → Keep or remove Site B recorded with resource cost
Cleanup / Restore
Snapshot: holodeck-06-complete
• Keep both sites or remove Site B with Remove-HoloDeckInstance from a session with the config imported
• Remove the Site B workstation route if Site B is removed
• Keep the RCAR and failure table for the design defence
Design Reflection (VCDX)
Multi-site is where candidates most often state wrong fundamentals. The lab lets you practise two management planes and inter-site routing; the defence requires you to say precisely what a stretched vSAN cluster needs (L2 or L3 between data sites, witness at a third site, preferred-site behaviour on partition), what NSX Federation adds and constrains (150 ms), how two instances become one fleet, and why shared infrastructure services (one router, one DNS) are a single failure domain. Separate what the lab proved from what only a real inter-site link can prove.
Requirements
- Two /20 CIDRs and two VLAN ranges with the documented minimum sizes
- HoloRouter in dual-site mode before either instance
- Inter-site L3 reachability and DNS for both suffixes
- Both management planes validated independently
- Design mapping with correct stretched-cluster and Federation facts
- Failure scenarios documented per the VCF documentation
- Snapshots before destructive steps
Constraints
- Dual-site resource row (for example 48 CPU / 768 GB / 2.5 TB for VCF 9.0 OSA with Automation)
- Custom values must be repeated on New-HoloDeckInstance
- Reset-HoloRouter affects both sites
- One physical host and one HoloRouter serve both sites
- Sub-millisecond lab latency is not representative
- Stretched vSAN: under 5 ms RTT between data sites, under 200 ms to the witness; Federation under 150 ms
- Three-host vSAN minimum per site
Assumptions
- Trunk carries both VLAN ranges
- Two SSH sessions available
- Capacity for two management-only instances
- Workstation routes for both /20s
- Binaries staged for 9.0.2.0
- Config imported in each session
- Holodeck 9.0.2 dual-site sequence as documented
- Both sites deployed management-only
Risks
- HoloRouter reboot — Impact: both sites lose routing and DNS — Mitigation: avoid -ResetHoloRouter while both run; snapshot first
- Concurrent deployments overload the host — Impact: nested crashes — Mitigation: sequential deployment or reduced options
- Custom networking overwritten by defaults — Impact: overlapping sites — Mitigation: repeat -CIDR/-VLANRangeStart on every cmdlet
- Wrong stretched-cluster facts in the defence — Impact: failed defence — Mitigation: Task 5 requirement list from the documentation
- Treating two instances as one fleet — Impact: wrong operational model — Mitigation: fleet integration as a separate exercise
- Version drift between sites — Impact: inconsistent estates — Mitigation: fleet-level lifecycle in production
- Witness placement error — Impact: no quorum on partition — Mitigation: third site requirement
Self-Assessment Discussion Prompts
- Why does the documented sequence configure both sites' networks and the router before any instance, and what happens if you change the order?
- What does a separate VRF per site give you in the lab, and what is its production analogue?
- State the vSAN stretched-cluster inter-site requirements and the behaviour on an inter-site partition.
- When would two instances in one fleet beat a stretched management cluster, and when the reverse?
- Which production constraints (latency, bandwidth, witness site) cannot be tested on one host, and how will you test them?
- How would you defend the shared HoloRouter as acceptable in a lab and unacceptable in production?
- What changes in identity (one broker) and security (Federation) when two instances must behave as one estate?
- How does version drift arise between two independently deployed sites, and what prevents it?
- Which failure scenario in your table is hardest to recover, and why?
- What is the smallest production footprint that satisfies the three-host minimum per site plus a witness?
Extensions
Integrate Site B into Site A's fleet
Follow vcp-admin-10 Task 2 to add the Site B instance to Site A's VCF Operations with a collector, then compare fleet-level operations across the two sites.
harderCustom CIDRs and VLAN ranges
Redeploy dual site with 10.3.0.0/20 and 10.4.0.0/20 and VLAN starts 100 and 200, repeating the values on every cmdlet as the documentation requires.
sameEdge clusters on both sites
Add -NsxEdgeClusterMgmtDomain to both deployments (capacity permitting) and verify two BGP peerings with the HoloRouter using Get-HoloDeckBGPConfig -Site a and -Site b.
harderCross-site SSO
Configure VCF SSO on Site A with an appliance identity broker and connect Site B's vCenter and NSX to it (vcp-admin-04); observe the cross-instance model.
harderNSX Federation reading
Read the NSX Federation section of the VCF documentation and write the design you would need for one DFW policy across both sites, including the 150 ms constraint.
sameSite B as VCF 5.2
Deploy Site B with -Version 5.2.2 to compare 5.2 and 9.0 management planes side by side (see holodeck-07 Task 4).
sameFailure-table rehearsal
For each row of your Task 5 failure table, write the exact VCF documentation section that supports the behaviour you describe.
same⚠ Known Pitfalls
References
- Holodeck 9.0.2 documentation (index): Environment Overview, Holorouter Overview, Concepts, Pre-requisites, Deployment, During Deployment, Post Deployment, Day 2 Operations, How ToTier 1 — Official
Primary source for every HoloRouter and toolkit step in this lab. Fetched 1 Oct 2026. - Holodeck 9.0.2 command referenceTier 1 — Official
New-HoloDeckInstance, Update-HoloDeckInstance, Get-HoloDeckInstance, New/Get/Import-HoloDeckConfig, Reset-HoloDeckState, New-HoloDeckNetworkConfig, Set-/Reset-HoloRouter, Get-HoloDeckSubnet/AppNetwork/BGPConfig/DNSConfig/OverlaySubnet/AppIpPools, Get-HolodeckServiceIPPools. - Holodeck 9.0.2 FAQ and troubleshootingTier 1 — Official
Pre-check errors (binary not found, no trunk port group, NTP not running), ESX build SSH wait, memory tiering, nested vSAN ESA. - Holodeck 9.0.2 downloads and release notes (known issues)Tier 1 — Official
HoloRouter OVA and Offline Depot OVA; release notes at /release_notes/ carry the Known Issues per release. - Holodeck GitHub repository: issues and discussionsTier 2 — Supporting source
Where to report bugs; community troubleshooting. The releases page carries no binaries (OVA downloads are on the Broadcom Support Portal). - VMware Cloud Foundation 9.0 documentation: Managing VCF Domains; Create a New Workload Domain Using VCF Operations; Commission ESX Hosts; vSAN cluster requirements; vSAN stretched clustersTier 1 — Official
VCF 9.0.1 PDF pp.467-468 (constructs), 1497-1505 (commission hosts, create workload domain wizard), 1663 (vSAN minimum three hosts), 1692-1695 and 1833 (stretched clusters). - VMware Cloud Foundation 9.0 documentation: NSX Federation (consistent firewall rules across locations; 150 ms maximum latency)Tier 1 — Official
VCF 9.0.1 PDF p.2289. Task 5. - VMware Cloud Foundation 9.0 documentation: vSAN stretched cluster network requirements and witness behaviourTier 1 — Official
VCF 9.0.1 PDF pp.1692-1695 ('vSAN supports both Layer 2 and Layer 3 between the two data sites'; 'vSAN continues operation with the preferred site'), p.1660 (witness at a third site), p.1833 Table 855. Task 5.