Holodeck 9.1 Click-to-Deploy GitOps — Part 1: HoloRouter OVA Deployment with GitLab Auto-Provisioning

VCF 9.1.0.0Intermediateadminarchitectvcdx⏱ 150 min

Written for Holodeck Toolkit 9.1 GA deploying VCF 9.1.0.0. The Toolkit also supports VVF 9.1.0.0 and the full range down to VCF 5.2 (including VCF 5.2.4) — the GitOps workflow in this lab is the same, only the content selection differs. Exact 9.1 Toolkit build number was not published in the release announcement; record it from your downloaded OVA and note it here (verify_live). This is the 9.1 parallel twin of holodeck-02; the 9.0.x cmdlet-driven track remains valid in holodeck-02 and is NOT replaced by this lab.

Objectives

Verify a physical ESXi host against the Holodeck 9.1 prerequisites (ESX 8.0 U3 minimum, sizing, VLAN and IP hygiene)
Deploy the Holodeck 9.1 HoloRouter OVA with GitOps enabled and observe the first-boot service stack initialization
Validate that the GitOps-enabled HoloRouter auto-provisions GitLab and exposes it behind the new HTTPS reverse proxy
Navigate the Click-to-Deploy UI, map its actions to Git commits, and locate the auto-generated Site A / Site B configuration structure
Author and commit a Site A VCF 9.1.0.0 deployment configuration entirely through the UI — without running any New-HoloDeck* cmdlet
Maintain a verify-live findings log that closes the gaps the 9.1 release announcement leaves open (OVF property names, bundled GitLab version, repository layout, initial credentials)

Prerequisites

Physical ESXi host at 8.0 U3 or later with Holodeck Toolkit 9.1 content staged and the 9.1 HoloRouter OVA downloaded and checksum-verified (holodeck91-01 completed). No existing 9.1 Holodeck instance on the host. Sizing baseline unchanged from the 9.0 track: minimum 512 GB RAM, 64 logical cores, 2 TB SSD/NVMe datastore. If the physical host also carries a 9.0.x Holodeck instance or a colleague's instance, coordinate before deploying — router-level operations on a shared host have historically been destructive to global router state (see Task 2 warnings).

Prior labs: holodeck91-01

Required skills:

ESXi host management and OVF/OVA deployment via vSphere Client
Basic Git concepts: repository, commit, branch, pipeline (taught lightly here, assumed at reading level)
Understanding of VCF management domain architecture (VCF Installer, SDDC Manager, vCenter, NSX)
Familiarity with the Holodeck 9.0.x cmdlet-driven deployment flow (holodeck-02) — this lab constantly contrasts against it
Reading TLS certificates in a browser (issuer, SAN, chain)

Lab Environment

Single physical ESXi host. This lab deploys ONLY the Holodeck 9.1 HoloRouter OVA with GitOps enabled. On first boot the router brings up its embedded service stack: GitLab (auto-provisioned for GitOps), HashiCorp Vault (:8200), Authentik SSO (:9443), Technitium DNS (:5380), authenticated Webtop (:30000) — all fronted by an HTTPS reverse proxy on the router's management IP. The nested VCF estate (ESX hosts, VCF Installer, SDDC Manager, vCenter, NSX) is NOT deployed in this lab; that is driven from the UI in holodeck91-03. The built-in services get their own deep dive in holodeck91-04.

graph TB
  WS[Operator Workstation] -->|HTTPS reverse proxy| HR[HoloRouter 9.1<br/>mgmt IP + 10.1.1.1 internal]
  HR --> GL[GitLab - auto-provisioned<br/>GitOps driver]
  HR --> VAULT[Vault :8200]
  HR --> AUTH[Authentik SSO :9443]
  HR --> DNS[Technitium DNS :5380]
  HR --> WT[Webtop :30000 - authenticated]
  GL -.->|holodeck91-03 triggers| NEST[Nested VCF 9.1.0.0 estate<br/>not yet deployed]

IP Addressing

Network	Purpose	VLAN
`10.1.1.0/20`	Management supernet for the future nested estate (mirrors the 9.0 track defaults — confirm the 9.1 UI defaults match before committing config)	VLAN 1644 (9.0-era default; 9.1 fixed VLAN-range handling bugs — verify defaults live)
`10.1.2.0/24`	vMotion (planned, deployed in holodeck91-03)	VLAN 1645 (verify live)
`10.1.3.0/24`	vSAN (planned, deployed in holodeck91-03)	VLAN 1646 (verify live)
`10.1.4.0/24`	NSX Host TEP (planned)	VLAN 1647 (verify live)
`10.1.5.0/24`	NSX Edge TEP (planned)	VLAN 1648 (verify live)

Credentials

System	Username	Password
Physical ESXi Host	`root`	Set during ESXi installation
HoloRouter 9.1 (SSH/console)	`admin`	Set via OVF properties at deploy time — record the property name in your findings log (verify_live)
GitLab (auto-provisioned)	`root`	Initial root password mechanism not documented in the release announcement — check OVF properties, first-boot console output, or a generated-secrets file on the router (verify_live). Record the retrieval path, never the password itself.
Authentik SSO	`akadmin (typical Authentik default — verify_live)`	Default credentials are an explicitly open question in the 9.1 changelog — resolve during deployment and record the mechanism
Webtop (:30000)	`holodeck (verify_live)`	Webtop is authenticated in 9.1 — determine whether it uses local credentials or Authentik SSO during this lab

Tasks

Task 1 Verify 9.1 prerequisites and stage the HoloRouter OVA

manageability

Holodeck 9.1 raises the minimum nested ESX to 8.0 U3 and changes the deployment driver — a pre-flight that was 'nice to have' in 9.0 is now mandatory because a GitOps pipeline will consume your configuration unattended. In production terms this is the Planning & Preparation Workbook checkpoint: garbage committed to Git becomes garbage deployed, repeatably.

Step 1

On the physical ESXi host, confirm the ESXi version meets the 9.1 floor. From the host client or SSH: vmware -vl

ESXi 8.0 Update 3 or later. The 9.1 release announcement states the minimum ESX version is 8.0 U3.

If the host is below 8.0 U3, stop and remediate before continuing. The 9.0 track tolerated older hosts in some scenarios; 9.1 explicitly does not.

Step 2

Confirm resource headroom: total RAM, logical cores, and free datastore capacity. From PowerCLI: Get-VMHost | Select-Object Name, MemoryTotalGB, NumCpu; Get-Datastore | Select-Object Name, FreeSpaceGB

At least 512 GB RAM, 64 logical cores, and more than 1.5 TB free on the target datastore, with less than 80% planned commitment after the nested estate lands in holodeck91-03.

The GitOps stack (GitLab + Vault + Authentik + Technitium + Webtop) runs ON the HoloRouter, so expect the 9.1 router appliance to be noticeably heavier than the 9.0 router. The exact resource overhead is an open question in the release notes — you will measure it in Task 3.

Step 3

Verify the Holodeck 9.1 content is staged for VCF 9.1.0.0 (from holodeck91-01) and note which additional trains you staged, if any (VVF 9.1.0.0, VCF 5.2.4).

Content directory contains the VCF 9.1.0.0 artifacts with verified checksums. The supported deployment range for Toolkit 9.1 is VCF 9.1.0.0 down to VCF 5.2.

Version mismatch between staged content and the version selected in the UI remains the most likely failure class — the 9.0-era KB is full of manifest/content mismatches (e.g. 'fail to deploy vcf 9.0.2.0'). The mechanism changed (UI selection vs config.json manifest path) but the failure class carries over.

Step 4

Check VLAN and IP hygiene on the physical host: list existing port groups and their VLAN IDs, and confirm the range you plan to use for the nested estate is free. PowerCLI: Get-VirtualPortGroup | Select-Object Name, VLanId

No existing port group conflicts with your planned VLAN range.

9.1 specifically fixed VLAN range handling and Site B port group bugs. If you were bitten by the 9.0-era 'Lab VLAN Range' conflict on a shared host, re-test rather than assume — the fix changes behavior, and your old workaround may now be unnecessary or even conflicting.

Step 5

Verify the HoloRouter 9.1 OVA checksum against the published value from the download source, and record the exact OVA file name and build string in your findings log.

Checksum matches. Findings log has its first entry: the concrete 9.1 build identifier (the announcement does not publish one).

Do not proceed with an unverified OVA. This appliance will hold secrets (Vault), identity (Authentik), and your deployment pipeline (GitLab) — supply-chain hygiene starts here.

Step 6

SHARED HOST CHECK: if any other Holodeck instance (yours or a colleague's) exists on this physical host, inventory it now and agree on ownership of router-level state before deploying.

Written note of existing instances, their IP ranges, and an agreed deployment window.

On the 9.0 track, Set-HoloRouter overwrote global router state (dnsmasq/FRR configuration) and broke co-tenants. Whether the 9.1 reverse-proxy + Technitium model isolates instances better is unverified — treat the shared-host scenario as dangerous until you have re-tested it under 9.1.

Validation Gate

Check: Confirm all five gates: ESXi >= 8.0 U3, resources within budget, VCF 9.1.0.0 content staged and checksummed, VLAN range clear, OVA checksum verified (plus shared-host agreement if applicable).

Expected: All gates pass. Findings log started with the concrete 9.1 build identifier.

Common Errors

Deployment later fails or behaves oddly on a host below ESXi 8.0 U3

Cause: 9.1 raised the minimum ESX version; older hosts that worked with the 9.0 Toolkit are no longer supported

Fix: Upgrade the physical host to 8.0 U3 or later before deploying the 9.1 OVA.

📋 KB: 9.1 release announcement — 'Minimum ESX Version: 8.0 U3'. No 9.1-era KB thread exists yet; the KB corpus is 9.0-era.

Colleague's Holodeck instance on the same host loses DNS/routing after your deployment

Cause: Router-level global state collision on a shared host (confirmed destructive on 9.0 via Set-HoloRouter; unverified under 9.1)

Fix: Revert your change, restore the colleague's router state from snapshot, and re-attempt only after agreeing a coordinated window. Log the observed 9.1 behavior — this is a known open question.

📋 KB: Internal note: Holodeck shared-holorouter destructiveness (9.0-era, re-validate on 9.1)

Task 2 Deploy the HoloRouter 9.1 OVA with GitOps enabled

manageability

This single OVF deployment replaces what the 9.0 track did across New-HoloDeckInstance's router staging: it is the 'Click' in Click-to-Deploy. Understanding exactly which OVF properties control GitOps enablement matters because they are the only imperative act left in the whole workflow — everything after this is declarative.

Step 1

In vSphere Client on the physical host, start Deploy OVF Template and select the verified HoloRouter 9.1 OVA.

OVF wizard opens and validates the OVA, showing the 9.1 appliance name and disk sizing.

Step 2

Select compute and the target datastore validated in Task 1. Use thin provisioning unless you have measured a reason not to.

Datastore accepted with ample free space remaining for the nested estate.

Step 3

Map networks: the router's external interface to your reachable management port group, and its internal/trunk interface(s) per the wizard's guidance.

Network mapping complete. External interface on a port group reachable from your workstation.

The exact interface names and count on the 9.1 appliance are not documented in the release announcement — follow the wizard's own descriptions and record the mapping in your findings log (verify_live). Do not blind-copy the 9.0 router's NIC layout; the service stack may have changed it.

Step 4

On the Customize Template page, locate and enable the GitOps option, and set the router admin credentials and management IP settings. Record the exact OVF property names and defaults in your findings log.

GitOps enablement property set to enabled; management IP, gateway, DNS, and admin password properties populated.

HONESTY GATE: the release announcement confirms the capability ('Deploy the HoloRouter OVA with GitOps enabled') but not the property name or default state. Verify against the live 9.1 toolkit during deployment — do not quote a property name in your design documentation that you have not seen on this page.

Screenshot the full Customize Template page. It is the authoritative record of every deploy-time knob in 9.1 and closes several release-notes gaps at once.

Step 5

Finish the wizard and power on the appliance. Open the VM console and watch first boot.

Appliance boots and begins service initialization. Expect a longer first boot than the 9.0 router — it is provisioning GitLab and the full service stack.

Note first-boot wall-clock time in your findings log. The GitOps appliance overhead is an explicitly open question from the release notes; your measurement is original data for the community.

Step 6

When the console settles, confirm basic reachability from your workstation: ping the router management IP, then ssh to it with the admin credentials you set.

Ping replies; SSH session established to the HoloRouter 9.1 appliance.

Validation Gate

Check: HoloRouter 9.1 VM is powered on, reachable by ping and SSH on its management IP, and your findings log records: OVF property names (especially the GitOps toggle), NIC mapping, and first-boot duration.

Expected: Router deployed with GitOps enabled and fully documented deploy-time configuration.

Common Errors

OVF deployment completes but the router never becomes reachable

Cause: External interface mapped to a port group that is not reachable from the workstation, or IP/gateway OVF properties mistyped

Fix: Check the VM console for the address it actually configured. Re-check port group VLAN and the OVF property values; redeploy if the management IP was wrong (faster than fixing in-guest on an appliance you don't know yet).

📋 KB: Failure class carried over from 9.0-era Thread 42 'Configuration of HoloRouter Failed' — mechanism differs in 9.1, symptom rhymes

First boot appears hung for a long time

Cause: GitLab and the service stack are provisioning on first boot; on slow storage this is lengthy. Alternatively the appliance is resource-starved.

Fix: Give it materially longer than a 9.0 router before declaring failure (measure, don't guess). Check the console for progress messages and the VM's CPU/memory demand in vSphere. If truly wedged, collect console output before redeploying — there is no 9.1 KB corpus yet, so your evidence is the KB.

📋 KB: No 9.1-era KB exists; document what you observe

Task 3 Validate GitLab auto-provisioning and the HTTPS reverse proxy

availability

The headline 9.1 claim is that enabling GitOps 'automatically spins up GitLab'. This task proves the claim in your environment and characterizes the two things the release notes leave open: the bundled GitLab version and its resource cost. It also establishes the reverse-proxy access pattern you will use for every service from now on.

Step 1

From your workstation browser, open https://<holorouter-mgmt-ip>/ and observe what the reverse proxy presents at the root path.

An HTTPS response from the router's reverse proxy — either a landing/portal page or a redirect to a service. Certificate is served by the proxy, not by individual services.

Record the root-path behavior. Whether 9.1 ships a portal page or a bare proxy is undocumented (verify_live).

Step 2

Locate the GitLab URL. Try the obvious candidates in order and record which resolves: a path or vhost on the reverse proxy, or a dedicated port. Do not guess in documentation — record only what worked.

GitLab sign-in page reachable over HTTPS via the reverse proxy.

HONESTY GATE: the announcement says GitLab is auto-provisioned and the UI drives deployments, but does not publish the GitLab URL scheme. Verify against the live 9.1 toolkit and log the working URL.

Step 3

Inspect the TLS certificate the reverse proxy serves (browser padlock > certificate). Record issuer, subject/SANs, and validity period in your findings log.

A certificate consistent with 'HTTPS everywhere' — likely appliance-generated. Note whether it chains to a root you could import (that workflow is exercised in holodeck91-04).

Compare with the 9.0 experience where several router services were plain HTTP. This is the concrete artifact of the 9.1 security overhaul.

Step 4

Retrieve the initial GitLab root credential using the mechanism you identified in Task 2 (OVF property, console output, or generated-secrets file on the router — whichever the live toolkit actually uses) and sign in.

Signed in to GitLab as root (or the appliance's equivalent bootstrap admin).

Record the retrieval MECHANISM in your findings log, never the secret value. If the mechanism is a flat file on the router, note that observation — it is directly relevant to the Vault discussion in holodeck91-04's design reflection.

Step 5

Record the GitLab version (Help/About page or /help) and the appliance resource demand right now (vSphere Client > HoloRouter VM > Monitor > Utilization).

Concrete GitLab version string and steady-state CPU/memory figures for the idle GitOps stack.

These are the two explicitly open questions from the structured changelog ('bundled GitLab version and resource overhead of the GitOps appliance'). You have now closed both for your environment.

Step 6

Confirm the other built-in services answer over HTTPS on their documented ports without deep-diving them yet: Vault https://<holorouter-mgmt-ip>:8200/v1/sys/health, Authentik https://<holorouter-mgmt-ip>:9443, Technitium https://<holorouter-mgmt-ip>:5380, Webtop https://<holorouter-mgmt-ip>:30000

All four endpoints respond over HTTPS. Vault's health endpoint returns JSON; Authentik and Technitium present login pages; Webtop presents an authentication challenge (it is no longer open access as in 9.0).

The four ports (8200, 9443, 5380, 30000) ARE documented in the release announcement — these you may quote confidently. The deep dive is holodeck91-04; here you only prove liveness.

Validation Gate

Check: GitLab reachable and signed in; TLS certificate details recorded; GitLab version and appliance resource figures logged; all four service ports (8200/9443/5380/30000) answering over HTTPS.

Expected: GitOps stack fully provisioned and characterized. Findings log closes the GitLab version and resource-overhead open questions for your environment.

Common Errors

GitLab sign-in page loads but bootstrap credentials are nowhere to be found

Cause: Initial credential mechanism differs from expectation (it is an undocumented area of the 9.1 release)

Fix: Check, in order: OVF properties you set at deploy time, the VM console first-boot output, and via SSH any generated-secrets location on the appliance. If all fail, reset via the appliance's documented GitLab rails console path — and write up whichever path worked.

📋 KB: No 9.1 KB yet — your writeup becomes the seed thread

Browser certificate warnings on every service

Cause: Reverse proxy serves an appliance-generated certificate not trusted by your workstation

Fix: Expected at this stage. Note the issuer now; the trust-import workflow (mirroring the 9.0 VMCA import from holodeck-04) is handled in holodeck91-04. Do not disable certificate validation globally.

📋 KB: Carry-over pattern from holodeck-04 Task 4 (9.0 track)

Vault health endpoint returns a 'sealed' status

Cause: Vault may initialize sealed depending on how the appliance bootstraps it — its unseal model in 9.1 is undocumented

Fix: Record the health JSON verbatim and continue; unseal handling is examined in holodeck91-04. For this lab, an HTTPS response of any well-formed status proves the service is provisioned.

📋 KB: verify_live — Vault bootstrap/unseal model is an open question

Task 4 Explore the Click-to-Deploy UI and the GitOps repository structure

manageability

GitOps means the repository is the source of truth and the UI is a front-end to commits. Before you trust a pipeline to build a full VCF estate (holodeck91-03), you must know where the truth lives: which repo, which files, which branch, and what a UI action looks like in Git history. A VCDX panelist will ask 'what exactly is your configuration record?' — this task is the answer.

Step 1

In GitLab, list the projects/groups the appliance auto-created. Record names and purposes in your findings log.

One or more auto-provisioned repositories related to Holodeck deployment (naming is undocumented — verify_live).

Look for separation between toolkit code and instance configuration. How 9.1 splits these determines your upgrade and rollback story.

Step 2

Open the configuration repository and inventory its structure: directories, file formats (JSON/YAML), and any pipeline definition file (e.g. .gitlab-ci.yml).

A browsable tree containing deployment configuration plus a CI pipeline definition that the Click-to-Deploy UI will trigger.

Record what IS there rather than what you expect. The 9.0 mental model (a single config.json under holodeck-runtime\templates) may map to multiple files here, or to a schema-validated document — the layout is verify_live.

Step 3

Locate the Site A and Site B configuration artifacts. The 9.1 release states dual-site configurations are auto-generated in one pass (the New-HoloDeckConfig behavior); confirm how the UI/repo represents both sites.

Both Site A and Site B configuration artifacts present (or a single artifact with both site sections).

You will deploy only Site A in holodeck91-03. Note precisely how Site B is parked (separate file? disabled flag?) — on a shared host, an accidental Site B deployment is an IP/VLAN collision waiting to happen with any existing instance.

Step 4

Find the Click-to-Deploy UI itself: the page from which a full Holodeck deployment is triggered. Candidates: a GitLab pipeline 'Run' page, a custom UI on the router, or a GitLab Pages front-end. Record which it is.

You can point at the exact screen where 'deploy' will be clicked in holodeck91-03.

HONESTY GATE: the announcement says deployments trigger 'directly from the UI' without specifying which UI. Verify against the live 9.1 toolkit and document the real entry point.

Step 5

Prove the UI-to-Git linkage: make one harmless, reversible change through the UI (for example, edit a description or comment field in the deployment configuration), save it, then inspect the repository's commit history.

A new commit appears, attributable to your UI action, with a diff showing exactly the field you changed.

This is the auditability demonstration in miniature. Note the commit author identity the UI uses — whether actions are attributed to a real user or a service account matters enormously for audit design (Authentik/SCIM ties into this in holodeck91-04).

Step 6

Revert your test change (via the UI if it supports it, otherwise via a Git revert) and confirm the history shows both the change and the revert.

Configuration back to original state, with a two-commit audit trail proving the round trip.

You have just executed the GitOps rollback primitive. Compare mentally with the 9.0 equivalent: editing config.json in Notepad, with no history at all.

Validation Gate

Check: Findings log contains: auto-created repo inventory, config repo structure, Site A/Site B artifact locations, the concrete Click-to-Deploy entry point, and a commit hash pair (change + revert) proving UI actions are Git commits.

Expected: You can state, with evidence, where the source of truth lives and how UI actions become auditable history.

Common Errors

Configuration repo looks empty or contains only pipeline scaffolding

Cause: Initial configuration may be generated on first UI use rather than at provisioning time

Fix: Open the Click-to-Deploy UI and start (but do not submit) a new deployment definition; re-check the repo for generated artifacts. Record the trigger point for generation.

📋 KB: verify_live — generation timing undocumented

UI change does not appear as a commit

Cause: The UI may stage changes in a draft state (database-backed) until an explicit save/apply action

Fix: Look for an explicit Save/Commit/Apply control in the UI and use it, then re-check history. If some UI state genuinely never reaches Git, document that carefully — it is a drift surface and belongs in your design reflection.

📋 KB: No 9.1 KB yet — significant finding if confirmed

Task 5 Author the Site A VCF 9.1.0.0 deployment configuration through the UI

manageability

This is the declarative equivalent of holodeck-02 Task 1 (reviewing config.json before a 2-hour build), with higher stakes: the committed configuration will drive an unattended pipeline in holodeck91-03. Every value you set here is a design decision with a Git-recorded justification opportunity.

Step 1

In the Click-to-Deploy UI, begin a new deployment configuration and select VCF 9.1.0.0 as the target version, referencing the content staged in holodeck91-01.

UI accepts the version selection and resolves it against staged content.

If the UI offers versions for which you have not staged content (it supports down to VCF 5.2), selecting one will fail late, not early — unless 9.1's hardened installer validation catches it. Note which it is when you see it.

Step 2

Configure the nested ESX host layout for the Site A management domain: host count and per-host CPU/memory sizing, mirroring the 9.0 baseline (4 hosts, 12 vCPU / 96 GB each) unless your physical host budget from Task 1 dictates smaller.

Host sizing accepted, totaling under 80% of physical capacity after accounting for the heavier 9.1 router appliance.

On the 9.0 track, editing sizing after a partial deployment only applied to the first host (a confirmed KB issue). Under GitOps the config is re-read from Git per pipeline run — whether that old caching bug class is structurally eliminated is worth testing later. For now: get sizing right BEFORE the first run, same discipline as 9.0.

Step 3

Configure networking for Site A: management CIDR (default 10.1.1.0/20 unless conflicting), VLAN assignments, and external reachability settings. Cross-check against the Task 1 VLAN inventory.

Network configuration consistent with the physical environment and free of conflicts with any co-tenant instance.

Your corporate network overlapping the nested CIDR remains the sneakiest failure mode (it survives from the 9.0 track unchanged): it causes intermittent routing weirdness, not clean errors. Change the CIDR now if your workstation network is anywhere in 10.0.0.0/8.

Step 4

Confirm Site B remains parked/disabled per your Task 4 finding. Do not remove its generated artifacts — the dual-site pair is the 9.1 design intent; you are deferring it, not deleting it.

Site A active for deployment; Site B present but inert.

Deleting Site B artifacts to 'clean up' would fight the toolkit's dual-site generation on the next config regeneration. Park, don't purge.

Step 5

Review every remaining field the UI exposes (passwords/secrets handling, snapshot options, Day-2 toggles like VCF Automation or Supervisor deployment). Set secrets carefully and note WHERE the UI stores them — this is a key observation for holodeck91-04's Vault discussion.

All fields reviewed and deliberately set. Findings log notes whether secrets appear committed to Git in plaintext, referenced from Vault, or handled elsewhere.

If you observe credentials being committed to Git in plaintext, flag it prominently in your findings log. If they are Vault-referenced, record the reference syntax. Either observation is defense-grade material.

Step 6

Save/commit the completed Site A configuration through the UI and verify the commit in the repository, reading the full diff end to end.

A single commit containing your complete, reviewed Site A configuration for VCF 9.1.0.0. The diff matches your intent exactly.

Write a meaningful commit message if the UI allows it (e.g. 'Site A VCF 9.1.0.0 mgmt domain - 4x12vCPU/96GB, 10.1.1.0/20, VLANs 1644-1648'). Your future self doing the holodeck91-03 post-mortem will thank you.

Validation Gate

Check: Committed Site A configuration exists in Git with: VCF 9.1.0.0 target, host sizing within physical budget, conflict-free networking, Site B parked, secrets handling observed and logged. You have read the full diff.

Expected: Deployment-ready declarative configuration, fully reviewed, with a documented understanding of where every value lives.

Common Errors

UI validation rejects the configuration with content/version errors

Cause: Selected VCF version does not match staged content, or content checksums fail — the hardened 9.1 installer validation catching it early (working as designed)

Fix: Re-verify the holodeck91-01 content staging for VCF 9.1.0.0. Prefer fixing content over overriding validation; the 9.1 hardening exists precisely because 9.0-era deployments failed late on this.

📋 KB: 9.0-era failure class (manifest mismatch threads); 9.1 changelog: 'hardened VCF installer validation'

Unsure whether an exotic UI field matters

Cause: 9.1 UI fields are not yet documented anywhere

Fix: Leave undocumented fields at defaults for the first deployment, and record the field names in your findings log. Change one variable at a time across deployment iterations.

📋 KB: verify_live discipline

Task 6 Snapshot the GitOps control plane and consolidate the findings log

recoverability

The router now carries your pipeline, secrets, identity, and DNS — it is the control plane for everything holodeck91-03 will build. Snapshotting it BEFORE the first pipeline run gives you a clean 'pipeline never ran' restore point, and the consolidated findings log is your personal 9.1 documentation until official docs and community KB catch up.

Step 1

Shut down cleanly or use a quiesce-capable snapshot: take a snapshot of the HoloRouter 9.1 VM named 'holodeck91-02-complete'. PowerCLI: Get-VM -Name '<holorouter-91-vm-name>' | New-Snapshot -Name 'holodeck91-02-complete' -Description 'GitOps stack provisioned, Site A config committed, no deployment run' -Memory:$false

Snapshot created on the router VM.

GitLab and Vault both dislike crash-consistent restores. Prefer a clean guest shutdown before snapshotting if your window allows; note which method you used so a future restore surprise is explainable.

Step 2

Verify the snapshot exists and boots: power the router back on (if shut down) and confirm GitLab still serves your committed Site A configuration.

Router back up; GitLab shows the Site A configuration commit at HEAD.

A snapshot you have not proven bootable is a hope, not a recovery plan.

Step 3

Consolidate the findings log into a single verify-live table with columns: [Open Question from Release Notes, What You Observed, Evidence (screenshot/commit/URL), Confidence]. Minimum rows: GitOps OVF property, GitLab URL scheme, GitLab version, appliance resource overhead, initial credentials mechanism, repo layout, Click-to-Deploy entry point, secrets handling in config.

Eight-plus rows of original 9.1 documentation grounded in your own evidence.

This table is the 9.1 twin of the 9.0 track's community-KB dependency: since zero 9.1 KB threads exist yet, you are writing the reference, not reading it.

Step 4

Cross-check your findings log against the structured changelog's section 7 open questions and mark which you have now closed for your environment (GitLab version, resource overhead) and which remain (VNA sizing, Technitium/dnsmasq coexistence, Authentik defaults, 9.0-to-9.1 upgrade path).

Explicit closed/open status per question. Technitium and Authentik questions carry into holodeck91-04.

Never mark a question closed on inference. Closed means you saw it in the live toolkit and captured evidence.

Validation Gate

Check: Snapshot 'holodeck91-02-complete' exists and is boot-verified. Findings table complete with evidence links. Open-question status reconciled against the changelog.

Expected: Clean restore point ahead of the first pipeline run, plus a defensible personal 9.1 reference document.

Common Errors

After snapshot revert, GitLab shows repository errors or Vault behaves unexpectedly

Cause: Crash-consistent snapshot captured GitLab/Vault mid-write

Fix: Prefer clean-shutdown snapshots for the GitOps control plane. If already bitten, use GitLab's built-in consistency checks and note Vault's seal state on boot. Record the failure mode — control-plane recoverability is a first-class design topic in holodeck91-04.

📋 KB: General GitLab/Vault operational practice; no 9.1 KB yet

Final Validation

The Holodeck 9.1 GitOps control plane is fully provisioned: HoloRouter 9.1 deployed with GitOps enabled, GitLab auto-provisioned and characterized (version, resource cost), all built-in services answering over HTTPS, and a reviewed Site A VCF 9.1.0.0 configuration committed to Git. No deployment has been triggered — that is holodeck91-03. A boot-verified snapshot protects the control plane, and a findings log documents every verify-live observation.

✓ HoloRouter 9.1 VM reachable via ping/SSH on management IP → Router up with GitOps stack running

✓ GitLab reachable over HTTPS via reverse proxy, signed in, version recorded → GitLab operational; version and resource overhead logged

✓ Service liveness: :8200 (Vault), :9443 (Authentik), :5380 (Technitium), :30000 (Webtop) all HTTPS → All four documented ports answering over HTTPS

✓ Site A VCF 9.1.0.0 configuration committed to Git, full diff reviewed → Single reviewed commit at HEAD; Site B parked, not deleted

✓ Snapshot 'holodeck91-02-complete' exists and boot-verified → Restore point proven before any pipeline run

✓ Findings log: 8+ verify-live rows with evidence → Release-notes open questions reconciled: closed vs carried forward

Cleanup / Restore

Snapshot: holodeck91-02-complete

• Snapshot the HoloRouter 9.1 VM as 'holodeck91-02-complete' (done in Task 6; prefer clean-shutdown snapshot)

• Store the findings log in your VCDX study folder alongside the Holodeck-9.1 reference docs — it is the only 9.1 field documentation in existence for your environment

• Sign out of GitLab and the service UIs; do not leave bootstrap admin sessions open

• If on a shared host, notify co-tenants that the 9.1 router is live and share its IP footprint

Design Reflection (VCDX)

A panelist handed this design would immediately probe the deployment-driver decision: 'You had a working cmdlet-based process in 9.0 — justify moving the driver into a GitOps pipeline.' Strong answers are architectural, not fashionable: the repository becomes the single source of truth, every configuration change is an attributable commit, and the pipeline makes deployment repeatable by construction rather than by operator discipline.
Equally, be ready for the inverse: the GitOps control plane (GitLab on the router) is new infrastructure with its own availability, recoverability, and security burden that the 9.0 PowerShell workstation simply did not have. If you cannot articulate what you TRADED for auditability — a heavier single-point-of-failure appliance, secrets concentration, unfamiliar failure modes with zero community KB — the panel will conclude you adopted the feature, not designed the solution.

Requirements

Deployment configuration must be version-controlled with attributable change history (who changed what, when)
A full Site A VCF 9.1.0.0 deployment must be triggerable without manual PowerShell on an operator workstation
The GitOps control plane must be restorable to a known-good state independent of the nested estate
All router-hosted services must be reachable only over HTTPS

Constraints

The 9.1 release announcement documents capabilities but not implementation details — every unlisted UI path, property, and credential mechanism must be field-verified before it enters design documentation
Community KB corpus is entirely 9.0-era: zero 9.1 threads exist, so there is no external validation for novel failure modes
GitLab, Vault, Authentik, Technitium, and Webtop all run on the single HoloRouter appliance — control-plane resource ceiling and failure domain are one VM
Physical host must be ESXi 8.0 U3 minimum (raised floor vs 9.0 track)
Shared-host deployments inherit the unresolved 9.0-era router-state collision risk until re-validated under 9.1

Assumptions

The 9.1 OVA's GitOps enablement provisions GitLab without external dependencies (no internet-dependent bootstrap) — verify on first deploy
PowerShell cmdlets remain available as a fallback deployment path per the release notes, even though this lab never uses them
Dual-site config auto-generation produces a Site B that is safely inert until explicitly deployed
Snapshot-based recovery of the router restores GitLab and Vault to a consistent state when taken from clean shutdown

Risks

Control-plane concentration: one appliance now carries pipeline, secrets, identity, and DNS — IMPACT: single failure removes the ability to deploy, authenticate, and resolve names simultaneously; MITIGATION: boot-verified snapshots, documented rebuild-from-OVA procedure, findings log as rebuild reference
Undocumented surface: acting on assumed UI paths or property names — IMPACT: misconfiguration or fabricated documentation that fails design defense scrutiny; MITIGATION: verify-live discipline, evidence-linked findings log
Secrets in Git: if deployment credentials are committed in plaintext the audit trail becomes a leak vector — IMPACT: anyone with repo read access holds the estate's keys; MITIGATION: observe and document actual behavior in Task 5, escalate to Vault-referenced secrets if supported (holodeck91-04)
Zero 9.1 KB: novel failures have no community answers — IMPACT: longer time-to-resolution than the KB-rich 9.0 track; MITIGATION: capture evidence rigorously and contribute findings back

Self-Assessment Discussion Prompts

Contrast the 9.0 cmdlet path (New-HoloDeckInstance against a hand-edited config.json) with the 9.1 GitOps path across three axes: repeatability, auditability, and drift. Which axis improves most, and which gains a new failure mode?
The 9.0 track's config.json had no history; the 9.1 config repo has full history. Give a concrete operational scenario from your own lab where that history would have saved you time, and one where it would not have helped at all.
The GitOps control plane runs ON the HoloRouter. Argue for and against separating it onto a dedicated appliance. What does production infrastructure practice say about co-locating the deployment system with the network gateway it deploys through?
PowerShell cmdlets remain available in 9.1. Under what circumstances would you deliberately choose the cmdlet path over the UI/GitOps path, and how would you keep Git as the source of truth if you did?
You observed (Task 4) how UI actions map to commits. If some UI state never reaches Git, what drift risks does that create, and how would you detect them?
A change made directly in Git (bypassing the UI) versus a change made in the UI: are both legitimate in a GitOps model? Design the change-control policy you would defend to a panel.

Extensions

Parallel Cmdlet-Path Deployment Comparison

On a second host (or after tearing down), perform the same router deployment using the 9.1 PowerShell cmdlet path that remains available, without GitOps enabled. Time both paths, diff the resulting router state, and document which built-in services exist in each mode. This directly answers the 'GitOps vs cmdlets' decision with primary evidence.

same

Characterize the GitOps Appliance Overhead Curve

Extend the Task 3 resource measurement into a time series: capture router CPU/memory at idle, during a config commit, and (after holodeck91-03) during a full pipeline run. Publish the curve in your findings log — this closes the release-notes open question quantitatively.

same

Break and Restore the Control Plane

After confirming the snapshot, deliberately corrupt the control plane (power-off mid-commit, fill the appliance disk) and practice restoring from 'holodeck91-02-complete'. Document GitLab and Vault behavior on restore from both clean-shutdown and crash-consistent snapshots.

harder

Shared-Host Coexistence Re-Validation

On a host carrying a 9.0.x instance, deploy the 9.1 router in a coordinated window and systematically test whether the 9.0-era global-state destructiveness (Set-HoloRouter overwriting shared dnsmasq/FRR state) has an analog under the 9.1 reverse-proxy/Technitium model. This resolves a standing open question for multi-tenant lab hosts.

much harder

⚠ Known Pitfalls (from Community KB)

[9.0-era, re-validate on 9.1] Configuration of HoloRouter Failed. Encountered error RESOLVED (9.0)

Problem: 9.0 router IP assignment failed during Prepare due to DHCP/port-group misconfiguration. The 9.1 OVA path replaces Prepare-phase router config with OVF properties, but the underlying port-group/VLAN failure class survives.

Resolution: Under 9.1: verify port group VLAN and OVF property values, watch the VM console for the actually-configured address, redeploy if wrong.

[9.0-era, re-validate on 9.1] Waiting for FRR service LIKELY_RESOLVED (9.0)

Problem: 9.0 Prepare hung on the router's FRR/BGP daemon. Whether the 9.1 appliance still runs FRR, and how its startup interleaves with the GitOps stack, is unverified.

Resolution: If first boot stalls, check routing daemon status via SSH before assuming GitLab provisioning is the culprit. Record which daemons the 9.1 appliance actually runs.

[9.1 open question] Bundled GitLab version and appliance resource overhead undocumented OPEN

Problem: The release announcement confirms GitLab auto-provisioning but not its version or resource cost; undersized hosts may starve the control plane.

Resolution: Measure both in Task 3 and log them. Budget the router as a heavyweight appliance, not a 9.0-class router.

[9.1 open question] Initial credential mechanisms (GitLab root, Authentik admin) undocumented OPEN

Problem: Bootstrap credential retrieval paths for the auto-provisioned services are not published.

Resolution: Check OVF properties, first-boot console, and on-appliance generated-secrets locations in that order; document the working mechanism, never the secret.

[Shared host] Router-level state collisions between co-tenant instances OPEN under 9.1

Problem: On 9.0, router reconfiguration overwrote global dnsmasq/FRR state and broke co-tenants. The 9.1 reverse-proxy/Technitium model may or may not isolate this.

Resolution: Coordinate deployment windows, snapshot co-tenant routers first, and re-validate the destructiveness question explicitly (see extension).

References

HoloDeck 9.1 Release Announcement (verbatim capture) Holodeck-9.1/01-release-announcement.mdlocal fileTier 1 — Official
Primary source for every capability claim in this lab: GitOps enablement, GitLab auto-provisioning, service ports, HTTPS reverse proxy, supported VCF range, minimum ESX 8.0 U3.
HoloDeck Toolkit 9.1 — Structured Reference & Changelog Holodeck-9.1/02-structured-changelog.mdlocal fileTier 1 — Official
Structured digest of the announcement including the section 7 open questions this lab's findings log is built around.
VCF 9.1 Release Intake — rn-9-1-016 (Click-to-Deploy GitOps) academy-v2/content/releases/vcf-9.1.jsonlocal fileTier 1 — Official
Release-intake entry confirming the deployment driver shift from New-HoloDeck* cmdlets to the UI/GitOps pipeline, with cmdlets remaining available.
VCF Holodeck Toolkit — Broadcom Community ForumTier 1 — Official
Community forum. NOTE: all 63 captured KB threads are 9.0-era; no 9.1 threads existed at authoring time. Treat carried-over fixes as hypotheses under 9.1.
GitLab CI/CD DocumentationTier 2 — VMware Press
Reference for reading the auto-provisioned pipeline definition (.gitlab-ci.yml) encountered in Task 4.
OpenGitOps PrinciplesTier 2 — VMware Press
The four GitOps principles (declarative, versioned and immutable, pulled automatically, continuously reconciled) — the evaluation framework used in the design reflection.
William Lam — VCF and Nested Lab AutomationTier 3 — Expert Blog
Community expert coverage of nested VCF deployment; watch for 9.1 Toolkit coverage as it emerges.

Mark complete