Academy/Holodeck Lab Setup & Operations/Holodeck 9.1 GitOps — Part 1: HoloRouter OVA Deployment with GitLab Auto-Provisioning
This lab targets VCF 9.1.0.0

Holodeck 9.1 GitOps — Part 1: HoloRouter OVA Deployment with GitLab Auto-Provisioning

VCF 9.1.0.0Intermediateadminarchitectvcdx⏱ 150 min

Written for Holodeck Toolkit 9.1 GA deploying VCF 9.1.0.0. The Toolkit also supports VVF 9.1.0.0 and the full range down to VCF 5.2 (including VCF 5.2.4) — the GitOps workflow in this lab is the same, only the content selection differs. Exact 9.1 Toolkit build number was not published in the release announcement; record it from your downloaded OVA and note it here (verify_live). This is the 9.1 parallel twin of holodeck-02; the 9.0.x cmdlet-driven track remains valid in holodeck-02 and is NOT replaced by this lab.

What changed in 9.1:

New 9.1-track twin of holodeck-02. With Enable GitOps selected at OVA deployment, HoloRouter installs and configures GitLab, creates a repository pre-loaded with the Holodeck deployment pipeline and registers itself as a GitLab runner, so a complete Holodeck deployment can be triggered from the GitLab UI (Holodeck 9.1 GA blog, 1 Jul 2026). The PowerShell cmdlets remain available. Part 1 covers control-plane bring-up and reading the pipeline; the Site A VCF 9.1.0.0 deployment is triggered from GitLab in holodeck91-03.

Objectives

  • Verify a physical ESXi host against the Holodeck 9.1 prerequisites (target host ESX 8.0 U3 or 9.0, documented 9.1 sizing, VLAN and IP hygiene)
  • Deploy the Holodeck 9.1 HoloRouter OVA with GitOps enabled and observe the first-boot service stack initialization
  • Validate that the GitOps-enabled HoloRouter auto-provisions GitLab and exposes it behind the new HTTPS reverse proxy
  • Open the pre-loaded Holodeck project in GitLab (https://gitlab.vcf.lab), read its pipeline definition, identify how the pipeline takes deployment inputs (variables vs committed files), and locate where the run executes (HoloRouter as the registered runner)
  • Record the Site A VCF 9.1.0.0 pipeline inputs you will use in holodeck91-03 and commit any input-file changes through GitLab — without running any New-HoloDeck* cmdlet
  • Maintain a verify-live findings log that closes the gaps the 9.1 release announcement leaves open (OVF property names, bundled GitLab version, repository layout, initial credentials)

Prerequisites

Physical ESXi host at 8.0 U3 or later with Holodeck Toolkit 9.1 content staged and the 9.1 HoloRouter OVA downloaded and checksum-verified (holodeck91-01 completed). No existing 9.1 Holodeck instance on the host. Holodeck 9.1 recommended sizing (docs): VCF 9.1 single site 48 CPU / 650 GB RAM / 2.2 TB SSD/NVMe (dual site 96 / 1536 GB / 5 TB); below this the pre-check warns but proceeds. If the physical host also carries a 9.0.x Holodeck instance or a colleague's instance, coordinate before deploying — the supported model is one HoloRouter per Holodeck instance, separated by -VLANRangeStart (maintainer, GitHub Discussion #178).

Prior labs: holodeck91-01

Required skills:

  • ESXi host management and OVF/OVA deployment via vSphere Client
  • Basic Git concepts: repository, commit, branch, pipeline (taught lightly here, assumed at reading level)
  • Understanding of VCF management domain architecture (VCF Installer, SDDC Manager, vCenter, NSX)
  • Familiarity with the Holodeck 9.0.x cmdlet-driven deployment flow (holodeck-02) — this lab constantly contrasts against it
  • Reading TLS certificates in a browser (issuer, SAN, chain)
📸 Starting State: S0-91 — Fresh Host (Holodeck 9.1)

Lab Environment

Single physical ESXi host. This lab deploys ONLY the Holodeck 9.1 HoloRouter OVA with GitOps enabled. On first boot the router brings up its embedded service stack: GitLab (auto-provisioned for GitOps), HashiCorp Vault (:8200), Technitium DNS (:5380), authenticated Webtop (:30000) — all fronted by an HTTPS reverse proxy on the router's management IP. Authentik SSO (:9443) is not there yet: it is installed by Set-HoloRouter, which runs as part of New-HoloDeckInstance (holodeck91-03). The nested VCF estate (ESX hosts, VCF Installer, SDDC Manager, vCenter, NSX) is NOT deployed in this lab; that is driven from the UI in holodeck91-03. The built-in services get their own deep dive in holodeck91-04.

Mermaid source
graph TB
  WS[Operator Workstation] -->|HTTPS reverse proxy| HR[HoloRouter 9.1<br/>mgmt IP + 10.1.1.1 Site A untagged gateway]
  HR --> GL[GitLab - auto-provisioned<br/>GitOps driver]
  HR --> VAULT[Vault :8200]
  HR -.-> AUTH[Authentik SSO :9443<br/>installed by Set-HoloRouter in holodeck91-03]
  HR --> DNS[Technitium DNS :5380]
  HR --> WT[Webtop :30000 - authenticated]
  GL -.->|holodeck91-03 triggers| NEST[Nested VCF 9.1.0.0 estate<br/>not yet deployed]

IP Addressing

NetworkPurposeVLAN
10.1.0.0/20Default Holodeck master CIDR for Site A (Site B 10.2.0.0/20); a custom CIDR must be a /20 (-CIDR). Per-VLAN subnets are carved from it: run Get-HoloDeckSubnet -Site a to list themSite A VLANs 0 and 10–25 (Site B 40–58); custom start via -VLANRangeStart (default [10,40])
Subnet carved from 10.1.0.0/20 (record from Get-HoloDeckSubnet -Site a)vMotion (planned, deployed in holodeck91-03)From the Site A range 10–25 (record from Get-HoloDeckSubnet -Site a)
Subnet carved from 10.1.0.0/20 (record from Get-HoloDeckSubnet -Site a)vSAN (planned, deployed in holodeck91-03)From the Site A range 10–25 (record from Get-HoloDeckSubnet -Site a)
Subnet carved from 10.1.0.0/20 (record from Get-HoloDeckSubnet -Site a)NSX Host TEP (planned)From the Site A range 10–25 (record from Get-HoloDeckSubnet -Site a)
Subnet carved from 10.1.0.0/20 (record from Get-HoloDeckSubnet -Site a)NSX Edge TEP (planned)From the Site A range 10–25 (record from Get-HoloDeckSubnet -Site a)

Credentials

SystemUsernamePassword
Physical ESX hostrootSet during ESX installation
HoloRouter 9.1 (SSH/console)rootSet in the OVA 'Password' property at deploy time (ssh root@<holorouter-ip>)
GitLab https://gitlab.vcf.lab (GitOps enabled)rootHolodeck documented default (Holodeck 9.1 docs, Accessing Holodeck Environment)
Authentik https://auth.vcf.labakadmin (Getting Started) or admin (release notes) — the docs conflict; record which one your build acceptsHolodeck documented default (Holodeck 9.1 docs, Accessing Holodeck Environment)
Webtop https://webtop.vcf.lab or http://<ip>:30000adminHolodeck documented default (Holodeck 9.1 docs, Accessing Holodeck Environment)
Technitium DNS https://dns.vcf.labadminHolodeck documented default (Holodeck 9.1 docs, Accessing Holodeck Environment)
Vault https://vault.vcf.lab(root token, no username)Holodeck documented default (Holodeck 9.1 docs, Accessing Holodeck Environment)

Tasks

Task 1 Verify 9.1 prerequisites and stage the HoloRouter OVA

manageability

Holodeck 9.1 keeps the ESX 8.0 U3 physical-host floor (8.0 U3 and 9.0 tested) and adds the GitOps deployment driver — a pre-flight that was 'nice to have' in 9.0 is now mandatory because a GitOps pipeline will consume your configuration unattended. In production terms this is the Planning & Preparation Workbook checkpoint: garbage committed to Git becomes garbage deployed, repeatably.

Step 1

On the physical ESXi host, confirm the ESXi version meets the 9.1 floor. From the host client or SSH: vmware -vl

ESXi 8.0 Update 3 or 9.0. The Holodeck docs list 'Minimum ESX Version: 8.0 U3' for the physical target host (vSphere 8.0u3 and 9.0 tested and supported).
If the host is below 8.0 U3, stop and remediate before continuing. The target-host floor (ESX 8.0 U3; 8.0 U3 and 9.0 tested) is the same as in Holodeck 9.0.x.
Step 2

Confirm resource headroom: total RAM, logical cores, and free datastore capacity. From PowerCLI: Get-VMHost | Select-Object Name, MemoryTotalGB, NumCpu; Get-Datastore | Select-Object Name, FreeSpaceGB

At or above the documented 9.1 sizing: 48 logical cores, 650 GB RAM and 2.2 TB free SSD/NVMe for single-site VCF 9.1 (soft validation warns below this), with less than 80% planned commitment after the nested estate lands in holodeck91-03.
The GitOps stack (GitLab + Vault + Authentik + Technitium + Webtop) runs ON the HoloRouter, so expect the 9.1 router appliance to be noticeably heavier than the 9.0 router. The exact resource overhead is an open question in the release notes — you will measure it in Task 3.
Step 3

Verify the Holodeck 9.1 content is staged for VCF 9.1.0.0 (from holodeck91-01) and note which additional trains you staged, if any (VVF 9.1.0.0, VCF 5.2.4).

Content directory contains the VCF 9.1.0.0 artifacts with verified checksums. The supported deployment range for Toolkit 9.1 is VCF 9.1.0.0 down to VCF 5.2.
Version mismatch between staged content and the version selected in the UI remains the most likely failure class — the 9.0-era KB is full of manifest/content mismatches (e.g. 'fail to deploy vcf 9.0.2.0'). The mechanism changed (UI selection vs config.json manifest path) but the failure class carries over.
Step 4

Check VLAN and IP hygiene on the physical host: list existing port groups and their VLAN IDs, and confirm the range you plan to use for the nested estate is free. PowerCLI: Get-VirtualPortGroup | Select-Object Name, VLanId

No existing port group conflicts with your planned VLAN range.
9.1 specifically fixed VLAN range handling and Site B port group bugs. If you were bitten by the 9.0-era 'Lab VLAN Range' conflict on a shared host, re-test rather than assume — the fix changes behavior, and your old workaround may now be unnecessary or even conflicting.
Step 5

Verify the HoloRouter 9.1 OVA checksum against the published value from the download source, and record the exact OVA file name and build string in your findings log.

Checksum matches. Findings log has its first entry: the concrete 9.1 build identifier (the announcement does not publish one).
Do not proceed with an unverified OVA. This appliance will hold secrets (Vault), identity (Authentik), and your deployment pipeline (GitLab) — supply-chain hygiene starts here.
Step 6

SHARED HOST CHECK: if any other Holodeck instance (yours or a colleague's) exists on this physical host, inventory it now and agree on ownership of router-level state before deploying.

Written note of existing instances, their IP ranges, and an agreed deployment window.
On the 9.0 track, Set-HoloRouter overwrote global router state (dnsmasq/FRR configuration) and broke co-tenants. Whether the 9.1 reverse-proxy + Technitium model isolates instances better is unverified — treat the shared-host scenario as dangerous until you have re-tested it under 9.1.

Validation Gate

Check: Confirm all five gates: ESXi >= 8.0 U3, resources within budget, VCF 9.1.0.0 content staged and checksummed, VLAN range clear, OVA checksum verified (plus shared-host agreement if applicable).

Expected: All gates pass. Findings log started with the concrete 9.1 build identifier.

Common Errors

Deployment later fails or behaves oddly on a host below ESXi 8.0 U3
Cause: The physical host is below the Holodeck floor (ESX 8.0 U3, unchanged since 9.0; 8.0u3 and 9.0 tested)
Fix: Upgrade the physical host to 8.0 U3 or later before deploying the 9.1 OVA.
📋 KB: Holodeck 9.1 documentation (supported-versions table) — minimum ESX 8.0 U3 for the physical target host. The captured community KB forum threads are 9.0-era; check the Holodeck GitHub issues for 9.1 reports.
Colleague's Holodeck instance on the same host loses DNS/routing after your deployment
Cause: Router-level global state collision on a shared host (confirmed destructive on 9.0 via Set-HoloRouter; unverified under 9.1)
Fix: Revert your change, restore the colleague's router state from snapshot, and re-attempt only after agreeing a coordinated window. Log the observed 9.1 behavior — this is a known open question.
📋 KB: Internal note: Holodeck shared-holorouter destructiveness (9.0-era, re-validate on 9.1)

Task 2 Deploy the HoloRouter 9.1 OVA with GitOps enabled

manageability

This single OVF deployment replaces what the 9.0 track did across New-HoloDeckInstance's router staging: it is what makes a GitLab-triggered deployment possible. Understanding exactly which OVF properties control GitOps enablement matters because they are the only imperative act left in the whole workflow — everything after this is declarative.

Step 1

In vSphere Client on the physical host, start Deploy OVF Template and select the verified HoloRouter 9.1 OVA.

OVF wizard opens and validates the OVA, showing the 9.1 appliance name and disk sizing.
Step 2

Select compute and the target datastore validated in Task 1. Use thin provisioning unless you have measured a reason not to.

Datastore accepted with ample free space remaining for the nested estate.
Step 3

Map networks: the router's external interface to your reachable management port group, and its internal/trunk interface(s) per the wizard's guidance.

Network mapping complete. External interface on a port group reachable from your workstation.
The exact interface names and count on the 9.1 appliance are not documented in the release announcement — follow the wizard's own descriptions and record the mapping in your findings log (verify_live). Do not blind-copy the 9.0 router's NIC layout; the service stack may have changed it.
Step 4

On the Customize Template page (the 9.1 Getting Started page calls it Additional settings), set the documented fields — Hostname, Management IP / CIDR / Gateway / DNS / NTP, DNS Domain and Password (the HoloRouter admin password) — and under Extra Properties confirm Enable GitOps (on by default) alongside Enable SSH and Enable UI (Webtop). Record the underlying OVF property keys and defaults you see in your findings log.

GitOps enablement property set to enabled; management IP, gateway, DNS, and admin password properties populated.
HONESTY GATE: the 9.1 release notes and GA announcement confirm the capability (HoloRouter OVA deployed with GitOps enabled), and the 9.1 Getting Started page shows the 'Enable GitOps' option as on by default, but the underlying OVF property name is not published. Verify against the live 9.1 toolkit during deployment — do not quote a property name in your design documentation that you have not seen on this page.
Screenshot the full Customize Template page. It is the authoritative record of every deploy-time knob in 9.1 and closes several release-notes gaps at once.
Step 5

Finish the wizard and power on the appliance. Open the VM console and watch first boot.

Appliance boots and begins service initialization. Expect a longer first boot than the 9.0 router — it is provisioning GitLab and the full service stack.
Note first-boot wall-clock time in your findings log. The GitOps appliance overhead is an explicitly open question from the release notes; your measurement is original data for the community.
Step 6

When the console settles, confirm basic reachability from your workstation: ping the router management IP, then ssh root@<holorouter-ip> with the Password you set on the Additional settings (Customize Template) screen in step 4.

Ping replies; SSH session established to the HoloRouter 9.1 appliance.

Validation Gate

Check: HoloRouter 9.1 VM is powered on, reachable by ping and SSH on its management IP, and your findings log records: OVF property names (especially the GitOps toggle), NIC mapping, and first-boot duration.

Expected: Router deployed with GitOps enabled and fully documented deploy-time configuration.

Common Errors

OVF deployment completes but the router never becomes reachable
Cause: External interface mapped to a port group that is not reachable from the workstation, or IP/gateway OVF properties mistyped
Fix: Check the VM console for the address it actually configured. Re-check port group VLAN and the OVF property values; redeploy if the management IP was wrong (faster than fixing in-guest on an appliance you don't know yet).
📋 KB: Failure class carried over from 9.0-era Thread 42 'Configuration of HoloRouter Failed' — mechanism differs in 9.1, symptom rhymes
First boot appears hung for a long time
Cause: GitLab and the service stack are provisioning on first boot; on slow storage this is lengthy. Alternatively the appliance is resource-starved.
Fix: Give it materially longer than a 9.0 router before declaring failure (measure, don't guess). Check the console for progress messages and the VM's CPU/memory demand in vSphere. If other services answer but GitLab does not, see the GH#139 entry in Task 3 before redeploying. If truly wedged, collect console output before redeploying and check the open Holodeck GitHub issues for a matching report.
📋 KB: Holodeck GitHub issues (github.com/vmware/Holodeck/issues), e.g. #139; document what you observe

Task 3 Validate GitLab auto-provisioning and the HTTPS reverse proxy

availability

The headline 9.1 claim is that enabling GitOps 'automatically spins up GitLab'. This task proves the claim in your environment and characterizes the two things the release notes leave open: the bundled GitLab version and its resource cost. It also establishes the reverse-proxy access pattern you will use for every service from now on.

Step 1

From your workstation browser, open https://<holorouter-mgmt-ip>/ and observe what the reverse proxy presents at the root path.

An HTTPS response from the router's reverse proxy — either a landing/portal page or a redirect to a service. Certificate is served by the proxy, not by individual services.
Record the root-path behavior. Whether 9.1 ships a portal page or a bare proxy is undocumented (verify_live).
Step 2

Locate the GitLab URL. Try the obvious candidates in order and record which resolves: a path or vhost on the reverse proxy, or a dedicated port. Do not guess in documentation — record only what worked.

GitLab sign-in page reachable over HTTPS via the reverse proxy.
The 9.1 Getting Started page documents GitLab at https://gitlab.vcf.lab (user root) and its registry at https://gitlab-registry.vcf.lab, resolved through HoloRouter DNS. Confirm the FQDN resolves and loads, and log what you see.
Step 3

Inspect the TLS certificate the reverse proxy serves (browser padlock > certificate). Record issuer, subject/SANs, and validity period in your findings log.

A certificate consistent with 'HTTPS everywhere' — likely appliance-generated. Note whether it chains to a root you could import (that workflow is exercised in holodeck91-04).
Compare with the 9.0 experience where several router services were plain HTTP. This is the concrete artifact of the 9.1 security overhaul.
Step 4

Sign in to GitLab (https://gitlab.vcf.lab) as root with the documented Holodeck default credential.

Signed in to GitLab as root; the pre-loaded Holodeck project is visible.
Change the default credentials if the lab is reachable from shared networks, and keep secrets out of your notes. The Vault discussion in holodeck91-04 covers where these services keep their secrets.
Step 5

Record the GitLab version (Help/About page or /help) and the appliance resource demand right now (vSphere Client > HoloRouter VM > Monitor > Utilization).

Concrete GitLab version string and steady-state CPU/memory figures for the idle GitOps stack.
These are two questions the Holodeck 9.1 documentation leaves open (bundled GitLab version and resource overhead of the GitOps appliance). You have now closed both for your environment.
Step 6

Confirm the other built-in services answer on their documented ports without deep-diving them yet: Vault https://<holorouter-mgmt-ip>:8200/v1/sys/health, Technitium https://<holorouter-mgmt-ip>:5380 (or https://dns.vcf.lab), Webtop https://webtop.vcf.lab through the reverse proxy (the direct port http://<holorouter-mgmt-ip>:30000 is plain HTTP and bypasses the proxy). Do not expect Authentik on :9443 yet: it is installed by Set-HoloRouter as part of New-HoloDeckInstance, so its check belongs to holodeck91-03/04. Record what you see on :9443 if you try it.

Vault and Technitium answer over HTTPS on their ports and Webtop over HTTPS at webtop.vcf.lab (or plain HTTP on :30000); record any of these that does not answer before New-HoloDeckInstance has run. Vault's health endpoint returns JSON; Technitium presents a login page; Webtop presents an authentication challenge (it is no longer open access as in 9.0).
The four ports (8200, 9443, 5380, 30000) ARE documented in the Holodeck 9.1 documentation service table — these you may quote confidently. Port 30000 is Webtop's direct HTTP port; HTTPS for Webtop is via the reverse-proxy FQDN. The deep dive is holodeck91-04; here you only prove liveness.

Validation Gate

Check: GitLab reachable and signed in; TLS certificate details recorded; GitLab version and appliance resource figures logged; Vault (8200) and Technitium (5380) answering over HTTPS (Authentik follows Set-HoloRouter in holodeck91-03), and Webtop answering over HTTPS at webtop.vcf.lab (direct :30000 is plain HTTP).

Expected: GitOps stack fully provisioned and characterized. Findings log closes the GitLab version and resource-overhead open questions for your environment.

Common Errors

GitLab rejects the documented root credential
Cause: The credential was changed, or the GitOps services have not finished first-boot provisioning
Fix: Wait for provisioning to finish (watch the HoloRouter console), then retry with the documented default; if it was changed, reset it from the GitLab rails console on HoloRouter.
📋 KB: Holodeck 9.1 docs — Accessing Holodeck Environment
Browser certificate warnings on every service
Cause: Reverse proxy serves an appliance-generated certificate not trusted by your workstation
Fix: Expected at this stage. Note the issuer now; the trust-import workflow (mirroring the 9.0 VMCA import from holodeck-04) is handled in holodeck91-04. Do not disable certificate validation globally.
📋 KB: Carry-over pattern from holodeck-04 Task 4 (9.0 track)
Vault health endpoint returns a 'sealed' status
Cause: Vault may initialize sealed depending on how the appliance bootstraps it — its unseal model in 9.1 is undocumented
Fix: Record the health JSON verbatim and continue; unseal handling is examined in holodeck91-04. For this lab, an HTTPS response of any well-formed status proves the service is provisioned.
📋 KB: verify_live — Vault bootstrap/unseal model is an open question
GitLab URL (https://gitlab.vcf.lab) unreachable after first boot while other services answer
Cause: Known Holodeck 9.1 issue GH#139: on first boot the HoloRouter iptables DROP rule sits above the port 80/443 ACCEPT rules
Fix: Apply the workaround in https://github.com/vmware/Holodeck/issues/139: on HoloRouter, move the port 80 and 443 ACCEPT rules above the DROP rule, then re-test. Check the issue thread for how to persist the change across reboots.
📋 KB: GitHub vmware/Holodeck #139 (label: Workaround provided)

Task 4 Explore the GitLab project and the Holodeck deployment pipeline

manageability

GitOps means the repository is the source of truth and GitLab is the front-end to it. Before you trust the pipeline to build a full VCF estate (holodeck91-03), you must know where the truth lives: which project, which files, which branch, how inputs reach the pipeline, and where it runs (HoloRouter is registered as the runner). A VCDX panelist will ask 'what exactly is your configuration record?' — this task is the answer.

Step 1

In GitLab, list the projects/groups the appliance auto-created. Record names and purposes in your findings log.

The pre-loaded Holodeck deployment project (created by HoloRouter when GitOps is enabled); record its name and group.
Look for separation between toolkit code and instance configuration. How 9.1 splits these determines your upgrade and rollback story.
Step 2

Open the configuration repository and inventory its structure: directories, file formats (JSON/YAML), and any pipeline definition file (e.g. .gitlab-ci.yml).

A browsable tree containing the pipeline definition (.gitlab-ci.yml) and the deployment inputs it reads.
Record what IS there rather than what you expect. The 9.0 mental model (a template at /holodeck-runtime/templates/config.json from which the cmdlets generate /holodeck-runtime/config/<config-ID>.json on HoloRouter) may map to multiple files here, or to a schema-validated document — the layout is verify_live.
Step 3

Identify how the pipeline receives deployment parameters (VCF version, instance ID, CIDR, vSAN mode, domain options): CI/CD variables entered at 'Run pipeline', or values committed in files. Record which, and where Site A / Site B are distinguished (-Site).

You can name the exact variables or files that set the Site A VCF 9.1.0.0 deployment, and how Site B would be parked.
You will deploy only Site A in holodeck91-03. Note precisely how Site B is parked (separate file? disabled flag?) — on a shared host, an accidental Site B deployment is an IP/VLAN collision waiting to happen with any existing instance.
Step 4

Locate the trigger: GitLab CI/CD > Pipelines > Run pipeline in the pre-loaded project, and confirm the registered runner is the HoloRouter (Settings > CI/CD > Runners, tags as set by the appliance).

You can point at the exact Run pipeline screen and the HoloRouter runner that will execute holodeck91-03.
The GA blog says deployments trigger directly from the GitLab UI; the pre-loaded project's Run pipeline page is that UI. Record the project, branch and runner tags you observe.
Step 5

Prove the Git linkage: make one harmless, reversible change in GitLab (for example, edit a comment line in an input file through the web editor), commit it, then inspect the commit history and diff.

A new commit appears, attributable to your UI action, with a diff showing exactly the field you changed.
This is the auditability demonstration in miniature. Note the commit author identity the UI uses — whether actions are attributed to a real user or a service account matters enormously for audit design (Authentik/SCIM ties into this in holodeck91-04).
Step 6

Revert your test change (via the UI if it supports it, otherwise via a Git revert) and confirm the history shows both the change and the revert.

Configuration back to original state, with a two-commit audit trail proving the round trip.
You have just executed the GitOps rollback primitive. Compare mentally with the 9.0 equivalent: a cmdlet-generated /holodeck-runtime/config/<config-ID>.json on HoloRouter, with no change history at all.

Validation Gate

Check: Findings log contains: auto-created repo inventory, config repo structure, Site A/Site B artifact locations, the concrete Run pipeline entry point, and a commit hash pair (change + revert) proving UI actions are Git commits.

Expected: You can state, with evidence, where the source of truth lives and how UI actions become auditable history.

Common Errors

Configuration repo looks empty or contains only pipeline scaffolding
Cause: Initial configuration may be generated on first UI use rather than at provisioning time
Fix: Open the pre-loaded project's Run pipeline page and review (but do not run) the inputs; re-check the repo for generated artifacts. Record the trigger point for generation.
📋 KB: verify_live — generation timing undocumented
UI change does not appear as a commit
Cause: The UI may stage changes in a draft state (database-backed) until an explicit save/apply action
Fix: Look for an explicit Save/Commit/Apply control in the UI and use it, then re-check history. If some UI state genuinely never reaches Git, document that carefully — it is a drift surface and belongs in your design reflection.
📋 KB: No matching community KB thread or Holodeck GitHub issue known at authoring time; check github.com/vmware/Holodeck/issues — significant finding if confirmed

Task 5 Record the Site A VCF 9.1.0.0 pipeline inputs

manageability

This is the declarative equivalent of holodeck-02 Task 1 (reviewing the cmdlet-generated deployment config before a 2-hour build), with higher stakes: the inputs you record here will drive an unattended pipeline run in holodeck91-03. Every value is a design decision with a Git-recorded justification opportunity.

Step 1

Decide and record the target version: VCF 9.1.0.0, referencing the content staged in holodeck91-01. If inputs are pipeline variables, prepare the variable list; if they are committed files, open the file you will edit.

Target version and input mechanism recorded in the findings log.
If the UI offers versions for which you have not staged content (it supports down to VCF 5.2), selecting one will fail late, not early — unless 9.1's hardened installer validation catches it. Note which it is when you see it.
Step 2

Record the Site A management-domain scope and options you will pass (management-only vs full stack, NSX Edge cluster, VCF Automation, Supervisor, vSAN mode). Holodeck sizes the nested hosts itself from the version profile (nested-host template default memory is 80 GB per the release notes); you do not author per-host sizing.

Deployment options recorded; physical commitment estimated under 80% including the heavier 9.1 HoloRouter.
On the 9.0 track, editing sizing after a partial deployment only applied to the first host (a confirmed KB issue). Under GitOps the config is re-read from Git per pipeline run — whether that old caching bug class is structurally eliminated is worth testing later. For now: get sizing right BEFORE the first run, same discipline as 9.0.
Step 3

Record networking for Site A: CIDR (default 10.1.0.0/20 unless conflicting; custom CIDRs must be /20) and -VLANRangeStart if the defaults collide with the physical environment. Cross-check against the Task 1 VLAN inventory.

Network configuration consistent with the physical environment and free of conflicts with any co-tenant instance.
Your corporate network overlapping the nested CIDR remains the sneakiest failure mode (it survives from the 9.0 track unchanged): it causes intermittent routing weirdness, not clean errors. Change the CIDR now if your workstation network is anywhere in 10.0.0.0/8.
Step 4

Confirm Site B is not part of this run (-Site a only). Leave Site B inputs untouched — dual site is deployed per the documented sequence (New-HoloDeckNetworkConfig per site, Set-HoloRouter -dualsite) in holodeck91-05/06.

Site A active for deployment; Site B present but inert.
Deleting Site B artifacts to 'clean up' would fight the toolkit's dual-site generation on the next config regeneration. Park, don't purge.
Step 5

Review every remaining field the UI exposes (passwords/secrets handling, snapshot options, Day-2 toggles like VCF Automation or Supervisor deployment). Set secrets carefully and note WHERE the UI stores them — this is a key observation for holodeck91-04's Vault discussion.

All fields reviewed and deliberately set. Findings log notes whether secrets appear committed to Git in plaintext, referenced from Vault, or handled elsewhere.
If you observe credentials being committed to Git in plaintext, flag it prominently in your findings log. If they are Vault-referenced, record the reference syntax. Either observation is defense-grade material.
Step 6

If inputs are committed files, commit the completed Site A inputs in GitLab and read the full diff end to end; if they are pipeline variables, save them as the project's CI/CD variables (masked where secret) and record the list.

A single commit (or saved variable set) containing your complete, reviewed Site A inputs for VCF 9.1.0.0.
Write a meaningful commit message (e.g. 'Site A VCF 9.1.0.0 mgmt-only, 10.1.0.0/20, default VLANs').

Validation Gate

Check: Committed Site A configuration exists in Git with: VCF 9.1.0.0 target, host sizing within physical budget, conflict-free networking, Site B parked, secrets handling observed and logged. You have read the full diff.

Expected: Deployment-ready declarative configuration, fully reviewed, with a documented understanding of where every value lives.

Common Errors

UI validation rejects the configuration with content/version errors
Cause: Selected VCF version does not match staged content, or content checksums fail — the hardened 9.1 installer validation catching it early (working as designed)
Fix: Re-verify the holodeck91-01 content staging for VCF 9.1.0.0. Prefer fixing content over overriding validation; the 9.1 hardening exists precisely because 9.0-era deployments failed late on this.
📋 KB: 9.0-era failure class (manifest mismatch threads); 9.1 release notes: 'hardened VCF Installer validation'
Unsure whether a pipeline variable matters
Cause: The pre-loaded pipeline's inputs are documented only in its own .gitlab-ci.yml
Fix: Leave undocumented variables at defaults for the first run, record the names in your findings log, change one variable at a time across iterations.
📋 KB: verify_live discipline

Task 6 Snapshot the GitOps control plane and consolidate the findings log

recoverability

The router now carries your pipeline, secrets, identity, and DNS — it is the control plane for everything holodeck91-03 will build. Snapshotting it BEFORE the first pipeline run gives you a clean 'pipeline never ran' restore point, and the consolidated findings log is your personal 9.1 documentation until official docs and community KB catch up.

Step 1

Shut down cleanly or use a quiesce-capable snapshot: take a snapshot of the HoloRouter 9.1 VM named 'holodeck91-02-complete'. PowerCLI: Get-VM -Name '<holorouter-91-vm-name>' | New-Snapshot -Name 'holodeck91-02-complete' -Description 'GitOps stack provisioned, Site A config committed, no deployment run' -Memory:$false

Snapshot created on the router VM.
GitLab and Vault both dislike crash-consistent restores. Prefer a clean guest shutdown before snapshotting if your window allows; note which method you used so a future restore surprise is explainable.
Step 2

Verify the snapshot exists and boots: power the router back on (if shut down) and confirm GitLab still serves your committed Site A configuration.

Router back up; GitLab shows the Site A configuration commit at HEAD.
A snapshot you have not proven bootable is a hope, not a recovery plan.
Step 3

Consolidate the findings log into a single verify-live table with columns: [Open Question from Release Notes, What You Observed, Evidence (screenshot/commit/URL), Confidence]. Minimum rows: GitOps OVF property, GitLab URL scheme, GitLab version, appliance resource overhead, initial credentials mechanism, repo layout, Run pipeline entry point, secrets handling in config.

Eight-plus rows of original 9.1 documentation grounded in your own evidence.
This table is the 9.1 twin of the 9.0 track's community-KB dependency: with the captured community threads 9.0-era and 9.1 reports mainly on the Holodeck GitHub issue tracker (e.g. GH#139), you are largely writing the reference, not reading it.
Step 4

Cross-check your findings log against the open questions this lab tracks and mark which you have now closed for your environment (GitLab version, resource overhead) and which remain (VNA sizing, Technitium/dnsmasq coexistence, Authentik defaults, 9.0-to-9.1 upgrade path).

Explicit closed/open status per question. Technitium and Authentik questions carry into holodeck91-04.
Never mark a question closed on inference. Closed means you saw it in the live toolkit and captured evidence.

Validation Gate

Check: Snapshot 'holodeck91-02-complete' exists and is boot-verified. Findings table complete with evidence links. Open-question status reconciled against the Holodeck 9.1 documentation and release notes.

Expected: Clean restore point ahead of the first pipeline run, plus a defensible personal 9.1 reference document.

Common Errors

After snapshot revert, GitLab shows repository errors or Vault behaves unexpectedly
Cause: Crash-consistent snapshot captured GitLab/Vault mid-write
Fix: Prefer clean-shutdown snapshots for the GitOps control plane. If already bitten, use GitLab's built-in consistency checks and note Vault's seal state on boot. Record the failure mode — control-plane recoverability is a first-class design topic in holodeck91-04.
📋 KB: General GitLab/Vault operational practice; no matching 9.1 KB thread or GitHub issue known at authoring time

Final Validation

The Holodeck 9.1 GitOps control plane is fully provisioned: HoloRouter 9.1 deployed with GitOps enabled, GitLab auto-provisioned and characterized (version, resource cost), all built-in services answering on their documented endpoints, and a reviewed Site A VCF 9.1.0.0 configuration committed to Git. No deployment has been triggered — that is holodeck91-03. A boot-verified snapshot protects the control plane, and a findings log documents every verify-live observation.

✓ HoloRouter 9.1 VM reachable via ping/SSH on management IP → Router up with GitOps stack running

✓ GitLab reachable over HTTPS via reverse proxy, signed in, version recorded → GitLab operational; version and resource overhead logged

✓ Service liveness: :8200 (Vault), :5380 (Technitium) over HTTPS (Authentik :9443 only after Set-HoloRouter in holodeck91-03); Webtop over HTTPS at webtop.vcf.lab (direct :30000 is plain HTTP) → All four documented services answering (three ports over HTTPS; Webtop HTTPS via webtop.vcf.lab)

✓ Site A VCF 9.1.0.0 configuration committed to Git, full diff reviewed → Single reviewed commit at HEAD; Site B parked, not deleted

✓ Snapshot 'holodeck91-02-complete' exists and boot-verified → Restore point proven before any pipeline run

✓ Findings log: 8+ verify-live rows with evidence → Release-notes open questions reconciled: closed vs carried forward

Cleanup / Restore

Snapshot: holodeck91-02-complete

• Snapshot the HoloRouter 9.1 VM as 'holodeck91-02-complete' (done in Task 6; prefer clean-shutdown snapshot)

• Store the findings log in your VCDX study folder alongside the Holodeck-9.1 reference docs — it is the only 9.1 field documentation in existence for your environment

• Sign out of GitLab and the service UIs; do not leave bootstrap admin sessions open

• If on a shared host, notify co-tenants that the 9.1 router is live and share its IP footprint

Design Reflection (VCDX)

A panelist handed this design would immediately probe the deployment-driver decision: 'You had a working cmdlet-based process in 9.0 — justify moving the driver into a GitOps pipeline.' Strong answers are architectural, not fashionable: the repository becomes the single source of truth, every configuration change is an attributable commit, and the pipeline makes deployment repeatable by construction rather than by operator discipline.
Equally, be ready for the inverse: the GitOps control plane adds GitLab, its registry and a CI runner to the same HoloRouter appliance that already hosted the Holodeck PowerShell module, DNS, DHCP and routing in 9.0, concentrating more state, secrets and availability burden in one VM. If you cannot articulate what you TRADED for auditability — a heavier single-point-of-failure appliance, secrets concentration, unfamiliar failure modes with a thin community KB (for example GH#139) — the panel will conclude you adopted the feature, not designed the solution.

Requirements

  • Deployment configuration must be version-controlled with attributable change history (who changed what, when)
  • A full Site A VCF 9.1.0.0 deployment must be triggerable without manual PowerShell on an operator workstation
  • The GitOps control plane must be restorable to a known-good state independent of the nested estate
  • All router-hosted services must be reachable only over HTTPS

Constraints

  • The 9.1 release announcement documents capabilities but not implementation details — every unlisted UI path, property, and credential mechanism must be field-verified before it enters design documentation
  • The captured community KB forum corpus is 9.0-era; for 9.1 the main external source is the Holodeck GitHub issue tracker (e.g. GH#139, GitLab unreachable on first boot), so external validation for novel failure modes is thin
  • GitLab, Vault, Authentik, Technitium, and Webtop all run on the single HoloRouter appliance — control-plane resource ceiling and failure domain are one VM
  • Physical host must be ESXi 8.0 U3 or 9.0 (same floor as the 9.0 track)
  • Shared-host deployments inherit the unresolved 9.0-era router-state collision risk until re-validated under 9.1

Assumptions

  • The 9.1 OVA's GitOps enablement provisions GitLab without external dependencies (no internet-dependent bootstrap) — verify on first deploy
  • PowerShell cmdlets remain available as a fallback deployment path per the release notes, even though this lab never uses them
  • Dual-site config auto-generation produces a Site B that is safely inert until explicitly deployed
  • Snapshot-based recovery of the router restores GitLab and Vault to a consistent state when taken from clean shutdown

Risks

  • Control-plane concentration: one appliance now carries pipeline, secrets, identity, and DNS — IMPACT: single failure removes the ability to deploy, authenticate, and resolve names simultaneously; MITIGATION: boot-verified snapshots, documented rebuild-from-OVA procedure, findings log as rebuild reference
  • Undocumented surface: acting on assumed UI paths or property names — IMPACT: misconfiguration or fabricated documentation that fails design defense scrutiny; MITIGATION: verify-live discipline, evidence-linked findings log
  • Secrets in Git: if deployment credentials are committed in plaintext the audit trail becomes a leak vector — IMPACT: anyone with repo read access holds the estate's keys; MITIGATION: observe and document actual behavior in Task 5, escalate to Vault-referenced secrets if supported (holodeck91-04)
  • Thin 9.1 KB: few novel failures have community answers beyond the Holodeck GitHub issues (e.g. GH#139) — IMPACT: longer time-to-resolution than the KB-rich 9.0 track; MITIGATION: capture evidence rigorously and contribute findings back

Self-Assessment Discussion Prompts

  1. Contrast the 9.0 cmdlet path (New-HoloDeckConfig and New-HoloDeckInstance on HoloRouter, writing /holodeck-runtime/config/<config-ID>.json) with the 9.1 GitOps path across three axes: repeatability, auditability, and drift. Which axis improves most, and which gains a new failure mode?
  2. The 9.0 track's cmdlet-generated /holodeck-runtime/config/<config-ID>.json files had no change history; the 9.1 config repo has full history. Give a concrete operational scenario from your own lab where that history would have saved you time, and one where it would not have helped at all.
  3. The GitOps control plane runs ON the HoloRouter. Argue for and against separating it onto a dedicated appliance. What does production infrastructure practice say about co-locating the deployment system with the network gateway it deploys through?
  4. PowerShell cmdlets remain available in 9.1. Under what circumstances would you deliberately choose the cmdlet path over the UI/GitOps path, and how would you keep Git as the source of truth if you did?
  5. You observed how pipeline inputs reach Git (committed files vs CI/CD variables). If some inputs never reach Git, what drift risks does that create, and how would you detect them?
  6. A change made directly in Git (bypassing the UI) versus a change made in the UI: are both legitimate in a GitOps model? Design the change-control policy you would defend to a panel.

Extensions

Parallel Cmdlet-Path Deployment Comparison

On a second host (or after tearing down), perform the same router deployment using the 9.1 PowerShell cmdlet path that remains available, without GitOps enabled. Time both paths, diff the resulting router state, and document which built-in services exist in each mode. This directly answers the 'GitOps vs cmdlets' decision with primary evidence.

same

Characterize the GitOps Appliance Overhead Curve

Extend the Task 3 resource measurement into a time series: capture router CPU/memory at idle, during a config commit, and (after holodeck91-03) during a full pipeline run. Publish the curve in your findings log — this closes the release-notes open question quantitatively.

same

Break and Restore the Control Plane

After confirming the snapshot, deliberately corrupt the control plane (power-off mid-commit, fill the appliance disk) and practice restoring from 'holodeck91-02-complete'. Document GitLab and Vault behavior on restore from both clean-shutdown and crash-consistent snapshots.

harder

Shared-Host Coexistence Re-Validation

On a host carrying a 9.0.x instance, deploy the 9.1 router in a coordinated window and systematically test whether the 9.0-era global-state destructiveness (Set-HoloRouter overwriting shared dnsmasq/FRR state) has an analog under the 9.1 reverse-proxy/Technitium model. This resolves a standing open question for multi-tenant lab hosts.

much harder

⚠ Known Pitfalls (from Community KB)

[9.0-era, re-validate on 9.1] Configuration of HoloRouter Failed. Encountered error RESOLVED (9.0)
Problem: 9.0 router IP assignment failed during Prepare due to DHCP/port-group misconfiguration. The 9.1 OVA path replaces Prepare-phase router config with OVF properties, but the underlying port-group/VLAN failure class survives.
Resolution: Under 9.1: verify port group VLAN and OVF property values, watch the VM console for the actually-configured address, redeploy if wrong.
[9.0-era, re-validate on 9.1] Waiting for FRR service LIKELY_RESOLVED (9.0)
Problem: 9.0 Prepare hung on the router's FRR/BGP daemon. Whether the 9.1 appliance still runs FRR, and how its startup interleaves with the GitOps stack, is unverified.
Resolution: If first boot stalls, check routing daemon status via SSH before assuming GitLab provisioning is the culprit. Record which daemons the 9.1 appliance actually runs.
[9.1 open question] Bundled GitLab version and appliance resource overhead undocumented OPEN
Problem: The release announcement confirms GitLab auto-provisioning but not its version or resource cost; undersized hosts may starve the control plane.
Resolution: Measure both in Task 3 and log them. Budget the router as a heavyweight appliance, not a 9.0-class router.
Authentik admin username differs between Holodeck 9.1 documents OPEN (documentation conflict)
Problem: The 9.1 Getting Started page gives akadmin; the 9.1 release notes give admin. The other service credentials (GitLab root, Webtop admin, Technitium admin, Vault root token) are documented.
Resolution: Try akadmin first, then admin; record which one your build accepts.
[Shared host] Router-level state collisions between co-tenant instances OPEN under 9.1
Problem: On 9.0, router reconfiguration overwrote global dnsmasq/FRR state and broke co-tenants. The 9.1 reverse-proxy/Technitium model may or may not isolate this.
Resolution: Coordinate deployment windows, snapshot co-tenant routers first, and re-validate the destructiveness question explicitly (see extension).

References

  • Announcing the General Availability of Holodeck 9.1 (VMware Cloud Foundation blog, 1 July 2026)Tier 1 — Official
    Official GA announcement: with GitOps enabled the HoloRouter installs and configures GitLab and registers itself as a GitLab runner; all HoloRouter services are HTTPS-only behind a built-in reverse proxy (authenticated Webtop on 30000, Authentik 9443, Technitium DNS 5380); supported range VCF 9.1.0.0 down to 5.2. It does not state the physical-host minimum — see the 9.1 documentation.
  • Holodeck 9.1 Documentation — Getting StartedTier 1 — Official
    Official 9.1 docs: supported-versions table (minimum ESX 8.0 U3), recommended physical-host sizing with soft validation, the 'Enable GitOps' OVA option (on by default) and the HoloRouter service table. Check this lab's open questions (bundled GitLab version, resource overhead) against what is documented here.
  • Holodeck 9.1 Release NotesTier 1 — Official
    Official 9.1 release notes: 'When the HoloRouter OVA is deployed with GitOps enabled, GitLab is automatically installed and configured on HoloRouter', with the deployment pipeline pre-loaded and the HoloRouter registered as a runner so a deployment can be triggered from the GitLab UI. The PowerShell cmdlets remain available (Getting Started).
  • VCF Holodeck Toolkit — Broadcom Community ForumTier 1 — Official
    Community forum. NOTE: all 63 captured KB threads are 9.0-era; no 9.1 threads existed at authoring time. Treat carried-over fixes as hypotheses under 9.1.
  • GitLab CI/CD DocumentationTier 2 — Supporting source
    Reference for reading the auto-provisioned pipeline definition (.gitlab-ci.yml) encountered in Task 4.
  • OpenGitOps PrinciplesTier 2 — Supporting source
    The four GitOps principles (declarative, versioned and immutable, pulled automatically, continuously reconciled) — the evaluation framework used in the design reflection.
  • William Lam — VCF and Nested Lab AutomationTier 3 — Background reading
    Community expert coverage of nested VCF deployment; watch for 9.1 Toolkit coverage as it emerges.
Was this page useful?
Type to search. ↑ ↓ to move, Enter to open, Esc to close.