Holodeck Instance Lifecycle Management
Objectives
- Design and execute a multi-checkpoint snapshot strategy for Holodeck lab environments
- Execute Remove-HoloDeckInstance and verify complete cleanup of nested infrastructure
- Redeploy from an existing config.json and measure redeploy time vs snapshot revert time
- Version-control config.json templates and demonstrate config overlay patterns for different VCF versions
- Deploy and compare side-by-side VCF 5.2.x and VCF 9.0.x instances to articulate version-specific architectural differences
- Automate Holodeck operations via PowerShell and document best practices for lab reset and scheduling
Prerequisites
Holodeck pod with VCF 9.0.2 management domain deployed and healthy (holodeck-02 completion snapshot available). Sufficient datastore space for 2+ snapshots per VM (minimum 200 GB free). PowerShell access to Holodeck Toolkit runtime directory.
Prior labs: holodeck-02
Required skills:
- Holodeck instance deployment and basic PowerShell scripting
- vSphere VM snapshot creation and revert operations
- Understanding of VCF component versioning and manifest structure
- File system navigation and JSON editing
Lab Environment
Primary Holodeck pod (VCF 9.0.2) from holodeck-02. Secondary instance (VCF 5.2.x) deployed in Task 4 on the same physical host using separate config. Both pods isolated via HoloRouter networking (Site-A: 10.1.1.0/20, Site-B: 10.10.0.0/20 if dual-site comparison is performed).
graph TB PHY[Physical ESXi Host 512+ GB RAM] --> HR[HoloRouter-A 10.1.1.1] PHY --> HR2[HoloRouter-B 10.10.0.1] HR --> MGMT-A[VCF 9.0.2 Instance] HR2 --> MGMT-B[VCF 5.2.x Instance] MGMT-A --> SNAP1[Snapshot: post-deploy] MGMT-A --> SNAP2[Snapshot: post-config] MGMT-A --> SNAP3[Snapshot: post-cleanup]
IP Addressing
| Network | Purpose | VLAN |
|---|---|---|
10.1.1.0/20 | Site-A primary instance (VCF 9.0.2) | VLAN 1644 |
10.10.0.0/20 | Site-B secondary instance (VCF 5.2.x, optional) | VLAN 1650 |
Credentials
| System | Username | Password |
|---|---|---|
| Physical ESXi Host | root | Configured during ESXi installation |
| HoloConsole-A | holodeck | Default Holodeck password from Toolkit docs |
| SDDC Manager | administrator@vsphere.local | From VCF bring-up spec. API authentication uses doubled password: VMware123!VMware123! |
| vCenter | administrator@vsphere.local | From VCF bring-up spec |
Tasks
Task 1 Design and execute a multi-checkpoint snapshot strategy
recoverabilitySnapshots are your lab reset button. In production, they're not allowed, but in a test environment, snapshots enable rapid rollback after experiments fail. VCDX panelists expect you to articulate when to snapshot (post-major-milestone), what to snapshot (which VMs), and what trade-offs exist (memory vs non-memory, disk overhead, consolidation timing). This task builds operational discipline.
Open PowerShell on the HoloConsole or your physical host management workstation. Set the working directory to the Holodeck runtime: cd C:\Holodeck\holodeck-runtime
Review the current instance state: Get-HoloDeckInstance | Format-Table -Property InstanceId, State, VcfVersion
List all Holodeck VMs currently deployed: Get-VM -Name 'Holo-*' | Format-Table -Property Name, MemoryMB, NumCpu, ProvisionedSpaceGB
Check datastore free space to estimate snapshot overhead: Get-Datastore | Where-Object {$_.Name -like 'holodeck'} | Format-Table -Property Name, FreeSpaceGB, CapacityGB
Create a non-memory snapshot at the post-deployment milestone: Get-VM -Name 'Holo-*' | New-Snapshot -Name 'lcm-post-deploy-9.0.2' -Description 'Post VCF 9.0.2 management domain deployment' -Memory:$false -Quiesce:$false
Verify snapshot creation: Get-VM -Name 'Holo-*' | Get-Snapshot | Format-Table -Property VM, Name, Created, SizeGB | Sort-Object Name
Now simulate a major lab change: perform a non-disruptive operation (e.g., document current SDDC Manager state). Then take a second checkpoint: Get-VM -Name 'Holo-*' | New-Snapshot -Name 'lcm-post-validation-9.0.2' -Description 'After operational validation' -Memory:$false -Quiesce:$false
Examine snapshot metadata including growth: $snapshots = Get-Snapshot -Name 'lcm-*'; $snapshots | Format-Table -Property VM, Name, SizeGB, Created
Review snapshot consolidation implications. Run: Get-VM -Name 'Holo-*' | Get-Snapshot | Measure-Object SizeGB -Sum
Document your naming convention in a text file: echo 'Snapshot Naming: <component>-<phase>-<version>. Examples: lcm-post-deploy-9.0.2, lcm-post-wld-9.0.2, lcm-post-config-9.0.2' > C:\Holodeck\holodeck-runtime\SNAPSHOT_NAMING.txt
Validation Gate
Check: Verify: (1) At least 2 snapshots exist per VM, (2) snapshot names follow the convention <component>-<phase>-<version>, (3) all snapshots are non-memory, (4) total snapshot size < 50% of free datastore space
Expected: Multi-checkpoint snapshot strategy in place with documented naming and growth tracking
Common Errors
Task 2 Destroy and redeploy an instance to verify cleanup and measure redeploy time
recoverabilityThe destroy-redeploy cycle is the ultimate validation that your instance is reproducible and that the Toolkit properly cleans up resources. VCDX panelists will probe: How do you ensure repeatability? Can you detect orphaned resources? How would you automate this in CI/CD? This task answers those questions.
Before destroying, document the instance configuration. Run: Get-HoloDeckInstance | ConvertTo-Json | Out-File C:\Holodeck\holodeck-runtime\instance-backup-before-destroy.json
Power off all Holodeck VMs to prepare for destruction. Run: Get-VM -Name 'Holo-*' | Stop-VM -Kill -Confirm:$false
Record pre-destruction datastore state: $datastorePre = Get-Datastore | Where-Object {$_.Name -like 'holodeck'}; $datastorePre | Format-Table -Property Name, FreeSpaceGB, CapacityGB
Execute instance removal. Get your instance ID first: $instanceId = (Get-HoloDeckInstance).InstanceId; Write-Host 'Destroying instance: '$instanceId; Remove-HoloDeckInstance -InstanceID $instanceId -Verbose
Verify cleanup: (a) No VMs named Holo- should exist: Get-VM -Name 'Holo-' -ErrorAction SilentlyContinue | Measure-Object
Verify (b) Port groups cleaned: Get-VirtualPortGroup | Where-Object {$_.Name -match '10\.0\.'} | Measure-Object
Verify (c) Datastore space reclaimed: $datastorePost = Get-Datastore | Where-Object {$_.Name -like 'holodeck'}; $spaceReclaimed = $datastorePost.FreeSpaceGB - $datastorePre.FreeSpaceGB; Write-Host 'Space reclaimed: '$spaceReclaimed' GB'
Now redeploy from scratch using the same config.json. First, prepare: $stopwatch = [System.Diagnostics.Stopwatch]::StartNew(); New-HoloDeckInstance -ConfigFile .\templates\config.json -Verbose
Start the full deployment: Start-HoloDeckInstance -InstanceID ((Get-HoloDeckInstance).InstanceId) -Verbose
Compare redeploy time vs snapshot revert time. Redeploy typically takes 90-150 minutes. Snapshot revert (if you had reverted instead of destroying) would take ~5-10 minutes. Document this in your lab notebook.
Validation Gate
Check: After destroy: (1) No VMs named Holo-* exist, (2) management port groups removed or minimal, (3) datastore space significantly recovered. After redeploy: (1) Instance returns to 'Running' state, (2) All management plane components healthy, (3) Redeploy time measured and logged
Expected: Complete lifecycle cycle demonstrated: destroy validates cleanup, redeploy validates reproducibility
Common Errors
Task 3 Version-control config.json templates and demonstrate config overlay patterns
manageabilityConfiguration management at scale: how do you manage multiple scenarios without copying and pasting? VCDX architectural thinking includes template reuse, inheritance, and override patterns. This task applies those principles to Holodeck configs.
Create a config directory structure for version control: mkdir -p C:\Holodeck\holodeck-runtime\configs\{base,overlays,deployed}
Copy your current config.json as the base template: Copy-Item .\templates\config.json .\configs\base\config-base-9.0.2.json
Create overlay configs for different scenarios. First, create an overlay for management-only deployment: @{ managementOnly = $true; vcfAutomationRequired = $false } | ConvertTo-Json | Out-File .\configs\overlays\overlay-mgmt-only.json
Create an overlay for different resource tiers (e.g., 3-host compact vs 4-host standard). Read the base config: $baseConfig = Get-Content .\configs\base\config-base-9.0.2.json | ConvertFrom-Json
Modify the config for a 3-host scenario (compact): $baseConfig.holodeck_sddc.'Site-A'.management.esxi.NumofNode = 3; $baseConfig.holodeck_sddc.'Site-A'.management.esxi.memory = 128; $baseConfig | ConvertTo-Json -Depth 20 | Out-File .\configs\deployed\config-vcf902-mgmt-3host.json
Verify the 3-host config: Get-Content .\configs\deployed\config-vcf902-mgmt-3host.json | ConvertFrom-Json | Select-Object -ExpandProperty holodeck_sddc | Select-Object -ExpandProperty 'Site-A' | Select-Object -ExpandProperty management | Select-Object -ExpandProperty esxi | Format-Table -Property NumofNode, memory
Create naming convention documentation: @'EOM
Holodeck Config Naming Convention:
---
(1) base configs: config-base-<vcf-version>.json
Example: config-base-9.0.2.json
(2) deployed configs: config-vcf<version>-<scenario>.json
Example: config-vcf902-mgmt-only.json
Example: config-vcf902-mgmt-3host.json
Example: config-vcf921-full-wld.json
(3) overlays: overlay-<feature>.json (for composition patterns)
Example: overlay-mgmt-only.json
Example: overlay-vsan-compact.json
(4) dated archives: config-<scenario>-<date>.json.bak
Example: config-vcf902-mgmt-only-2026-04-17.json.bak
EOM
' | Out-File .\configs\CONFIG_NAMING.md
Create a version history file: Add-Content C:\Holodeck\holodeck-runtime\configs\VERSION_HISTORY.txt -Value @"
2026-04-17: config-vcf902-mgmt-only.json deployed successfully (90min)
2026-04-17: config-vcf902-mgmt-3host.json validated (85min)
2026-04-16: config-base-9.0.2.json created as baseline
"@
Demonstrate config validation before deployment. Create a validation function in PowerShell: function Validate-HoloDeckConfig { param([string]$ConfigFile); $config = Get-Content $ConfigFile | ConvertFrom-Json; if (-not $config.vcf.version) { Write-Host 'ERROR: vcf.version not set' } else { Write-Host 'VCF version: '$config.vcf.version }; if (-not $config.network.management_cidr) { Write-Host 'ERROR: management_cidr not set' } else { Write-Host 'Management CIDR: '$config.network.management_cidr } }; Validate-HoloDeckConfig -ConfigFile .\configs\deployed\config-vcf902-mgmt-only.json
Create a config snapshot feature: function Export-ConfigSnapshot { param([string]$ConfigId); $timestamp = Get-Date -Format 'yyyyMMdd-HHmmss'; Copy-Item .\configs\deployed\$ConfigId .\configs\base\archive\$ConfigId'.snapshot-'$timestamp'.json }; Export-ConfigSnapshot -ConfigId 'config-vcf902-mgmt-only.json'
Validation Gate
Check: Verify: (1) Base config exists at configs/base/, (2) At least 2 deployed configs at configs/deployed/, (3) Naming convention documented, (4) Version history log populated, (5) Validation function works correctly
Expected: Config management framework in place with templates, overlays, naming convention, and version history
Common Errors
Task 4 Deploy side-by-side VCF 5.2.x and 9.0.x instances and document version differences
manageabilityThis is VCDX gold. Demonstrating deep knowledge of version differences shows architectural maturity. By running 5.2.x and 9.0.x simultaneously, you can articulate what changed between releases (component versions, bring-up sequence, SDDC Manager UI, feature set). Panelists will probe your design choices when considering upgrades.
Verify your VCF 5.2.x content manifest is available. List content manifests: Get-ChildItem .\content\manifests\ | Where-Object {$_.Name -like '5.2*'} | Format-Table Name
Create a VCF 5.2.x config by copying the base config and modifying version: $config521 = Get-Content .\configs\base\config-base-9.0.2.json | ConvertFrom-Json; $config521.vcfVersion = '5.2.1'; $config521.contentManifest = '.\content\manifests\vcf-5.2.1-manifest.json'; $config521 | ConvertTo-Json -Depth 20 | Out-File .\configs\deployed\config-vcf521-mgmt-only.json
Update the Site-A naming in the config to avoid conflicts with the 9.0.2 instance. Modify the site identifier: $config521.holodeck_sddc | Add-Member -MemberType NoteProperty -Name 'Site-B' -Value $config521.holodeck_sddc.'Site-A' -Force; $config521.holodeck_sddc.PSObject.Properties.Remove('Site-A'); $config521 | ConvertTo-Json -Depth 20 | Out-File .\configs\deployed\config-vcf521-mgmt-only.json
Prepare the VCF 5.2.1 instance: New-HoloDeckInstance -ConfigFile .\configs\deployed\config-vcf521-mgmt-only.json -Verbose
Start the VCF 5.2.1 deployment in parallel (or sequentially if resources are limited): Start-HoloDeckInstance -InstanceID ((Get-HoloDeckInstance | Where-Object {$_.VcfVersion -eq '5.2.1'}).InstanceId) -Verbose
Once 5.2.1 deployment completes, validate both instances. Get the VCF 5.2.1 SDDC Manager IP (different Site-B network) and compare: Write-Host 'VCF 9.0.2 instances:'; Get-HoloDeckInstance | Where-Object {$_.VcfVersion -eq '9.0.2'} | Format-Table -Property InstanceId, State, VcfVersion; Write-Host 'VCF 5.2.1 instances:'; Get-HoloDeckInstance | Where-Object {$_.VcfVersion -eq '5.2.1'} | Format-Table -Property InstanceId, State, VcfVersion
Access both SDDC Managers and document differences. For 5.2.x, navigate to https://10.10.0.4 (Site-B network). For 9.0.2, navigate to https://10.1.1.4 (Site-A network). Compare: (a) SDDC Manager UI layout and menu structure, (b) Component versions listed in System > About, (c) Workload Domain configuration workflow
Compare component versions. Access vCenter on both instances and check: Administration > System Configuration > Servers. Document the vCenter version for each instance (typically vCenter 7.x for 5.2.x, vCenter 8.x for 9.0.x).
Document 5+ specific architectural or operational differences. Create a comparison document: @'EOM
VCF 5.2.1 vs VCF 9.0.2 Differences:
---
- SDDC Manager UI:
5.2.x: Simplified dashboard, fewer Day 2 operations visible
9.0.x: Enhanced UI with VCF Automation integration, expanded Day 2 menu
- vCenter Version:
5.2.x: vCenter 7.0.x
9.0.x: vCenter 8.0.x
- NSX Version:
5.2.x: NSX 4.x
9.0.x: NSX 9.0.x
- Cloud Builder Role:
5.2.x: Cloud Builder required for bring-up, then can be removed
9.0.x: Cloud Builder retained for SDDC lifecycle operations
- Licensing Model:
5.2.x: Older licensing mechanism, limited evaluation trial
9.0.x: New licensing portal integration, extended evaluation trial
EOM
' | Out-File C:\Holodeck\holodeck-runtime\COMPARISON_5.2-vs-9.0.md
Create a summary discussing VCDX-relevant insights: 'From a design perspective, VCF 9.0.x introduces tighter integration between SDDC Manager and lifecycle operations. This affects how you plan upgrade windows, capacity planning, and automation. A VCDX candidate should understand these version-specific constraints when designing infrastructure.'
Validation Gate
Check: Verify: (1) Two instances deployed (VCF 5.2.1 and 9.0.2), both in Running state, (2) Both SDDC Managers accessible, (3) Component version comparison table created, (4) At least 5 documented differences, (5) VCDX design reflection written
Expected: Side-by-side comparison complete with architectural insights documented for panelist discussion
Common Errors
Task 5 Automate Holodeck operations and build a lab reset script
manageabilityOperational maturity means repeatability and automation. VCDX-level thinking includes building frameworks that others (or your future self) can execute without manual intervention. This task applies software engineering best practices (idempotency, error handling, logging) to lab operations.
Create a PowerShell module for common Holodeck operations. Create file: C:\Holodeck\holodeck-runtime\HoloDeckOps.psm1
Add a function to deploy an instance with logging: @'
function New-HoloDeckDeployment {
param(
[string]$ConfigFile,
[string]$LogPath = 'C:\Holodeck\holodeck-runtime\logs'
)
$timestamp = Get-Date -Format 'yyyyMMdd-HHmmss'
$logFile = Join-Path $LogPath "deployment-$timestamp.log"
Write-Host "[$(Get-Date)] Starting deployment with config: $ConfigFile" | Tee-Object -FilePath $logFile -Append
try {
New-HoloDeckInstance -ConfigFile $ConfigFile -Verbose 2>&1 | Tee-Object -FilePath $logFile -Append
$instanceId = (Get-HoloDeckInstance | Where-Object {$_.State -eq 'Prepared'}).InstanceId
Write-Host "[$(Get-Date)] Prepared instance: $instanceId" | Tee-Object -FilePath $logFile -Append
Start-HoloDeckInstance -InstanceID $instanceId -Verbose 2>&1 | Tee-Object -FilePath $logFile -Append
Write-Host "[$(Get-Date)] Deployment completed successfully" | Tee-Object -FilePath $logFile -Append
return $instanceId
}
catch {
Write-Host "[$(Get-Date)] ERROR: $_" | Tee-Object -FilePath $logFile -Append
throw
}
}
export-modulemember -function New-HoloDeckDeployment
' | Out-File C:\Holodeck\holodeck-runtime\HoloDeckOps.psm1 -Encoding UTF8
Add a snapshot management function: @'
function New-HoloDeckCheckpoint {
param(
[string]$CheckpointName,
[string]$Description
)
Write-Host "Creating checkpoint: $CheckpointName"
Get-VM -Name 'Holo-*' | New-Snapshot -Name $CheckpointName -Description $Description -Memory:$false -Quiesce:$false | Out-Null
Write-Host "Checkpoint created: $CheckpointName"
}
function Revert-HoloDeckCheckpoint {
param(
[string]$CheckpointName
)
Write-Host "Reverting to checkpoint: $CheckpointName"
Get-VM -Name 'Holo-*' | Get-Snapshot -Name $CheckpointName | Foreach-Object { Set-VM -VM $_.VM -Snapshot $_ -Confirm:$false } 2>&1
Write-Host "Checkpoint reverted: $CheckpointName"
}
export-modulemember -function New-HoloDeckCheckpoint, Revert-HoloDeckCheckpoint
' | Add-Content C:\Holodeck\holodeck-runtime\HoloDeckOps.psm1 -Encoding UTF8
Create a lab reset script: @'
HoloDeck Lab Reset Script
Usage: .\Reset-Lab.ps1 -TargetSnapshot 'holodeck-02-complete'
param(
[string]$TargetSnapshot = 'holodeck-02-complete',
[switch]$Confirm = $true
)
Import-Module C:\Holodeck\holodeck-runtime\HoloDeckOps.psm1
Write-Host "Lab Reset: Reverting all VMs to snapshot: $TargetSnapshot"
if ($Confirm) {
$response = Read-Host "Confirm reset? (yes/no)"
if ($response -ne 'yes') { Write-Host 'Cancelled'; exit }
}
Get-VM -Name 'Holo-*' | Where-Object { Get-Snapshot -VM $_ -Name $TargetSnapshot -ErrorAction SilentlyContinue } | Foreach-Object {
Write-Host "[$(Get-Date)] Reverting: $($_.Name)"
Get-Snapshot -VM $_ -Name $TargetSnapshot | Set-VM -Snapshot -Confirm:$false | Out-Null
}
Write-Host "[$(Get-Date)] Lab reset complete. Starting VMs..."
Get-VM -Name 'Holo-*' | Start-VM -Confirm:$false | Out-Null
Write-Host "Lab ready"
' | Out-File C:\Holodeck\holodeck-runtime\Reset-Lab.ps1 -Encoding UTF8
Create a scheduled snapshot script. This script runs weekly to create a clean checkpoint: @'
Scheduled Weekly Snapshot
This runs every Friday at 18:00 via Windows Task Scheduler
$timestamp = Get-Date -Format 'yyyyMMdd'
$checkpointName = "auto-weekly-$timestamp"
Write-Host "[$(Get-Date)] Creating weekly checkpoint: $checkpointName"
Get-VM -Name 'Holo-*' | New-Snapshot -Name $checkpointName -Description "Auto-generated weekly snapshot" -Memory:$false -Quiesce:$false | Out-Null
Write-Host "[$(Get-Date)] Snapshot created"
Clean up snapshots older than 4 weeks
$cutoffDate = (Get-Date).AddDays(-28)
Get-VM -Name 'Holo-*' | Get-Snapshot | Where-Object { $_.Created -lt $cutoffDate } | Remove-Snapshot -Confirm:$false
Write-Host "[$(Get-Date)] Old snapshots cleaned up"
' | Out-File C:\Holodeck\holodeck-runtime\ScheduledSnapshot.ps1 -Encoding UTF8
Test the reset script (dry-run): Read the script to verify syntax, then test on a single VM: Get-VM -Name 'Holo-Console' | Get-Snapshot | Measure-Object
Create a CI/CD integration example. This script validates a config before deployment: @'
function Test-HoloDeckConfig {
param([string]$ConfigFile)
Write-Host "Validating config: $ConfigFile"
$config = Get-Content $ConfigFile | ConvertFrom-Json
Validation checks
$checks = @(
('vcfVersion defined', -not [string]::IsNullOrEmpty($config.vcfVersion)),
('contentManifest exists', (Test-Path $config.contentManifest)),
('ESXi count > 2', $config.holodeck_sddc."Site-A".management.esxi.NumofNode -ge 2)
)
$passed = 0
foreach ($check in $checks) {
$status = if ($check[1]) { 'PASS' } else { 'FAIL' }
Write-Host " [$status] $($check[0])"
if ($check[1]) { $passed++ }
}
return ($passed -eq $checks.Count)
}
export-modulemember -function Test-HoloDeckConfig
' | Out-File C:\Holodeck\holodeck-runtime\HoloDeckValidation.psm1 -Encoding UTF8
Create a deployment status dashboard script. This shows health of all instances: @'
Write-Host "Holodeck Deployment Status"
Write-Host "============================="
Get-HoloDeckInstance | Format-Table -Property @(
'InstanceId',
'State',
'VcfVersion',
@{Label='SDDC Manager';Expression={Get-VM -Name 'SDDC-Manager*' -ErrorAction SilentlyContinue | Measure-Object | Select-Object -ExpandProperty Count}}
) -AutoSize
Write-Host ""
Write-Host "Datastore Usage"
Write-Host "==============="
Get-Datastore | Where-Object {$_.Name -like 'holodeck'} | Format-Table -Property Name, @{Label='Free GB';Expression={[Math]::Round($_.FreeSpaceGB, 0)}}, @{Label='Total GB';Expression={[Math]::Round($_.CapacityGB, 0)}} -AutoSize
Write-Host ""
Write-Host "Snapshots"
Write-Host "========="
Get-VM -Name 'Holo-*' | Get-Snapshot | Group-Object -Property VM | Format-Table -Property Name, @{Label='Snapshot Count';Expression={$_.Count}} -AutoSize
' | Out-File C:\Holodeck\holodeck-runtime\Show-LabStatus.ps1 -Encoding UTF8
Test the modular functions: Import-Module C:\Holodeck\holodeck-runtime\HoloDeckOps.psm1; New-HoloDeckCheckpoint -CheckpointName 'test-automation' -Description 'Testing automation script'
Document the automation framework in a README: @'
Holodeck Automation Framework
Available Functions
Deployment
New-HoloDeckDeployment -ConfigFile <path>: Deploy instance with logging
Checkpoints
New-HoloDeckCheckpoint -CheckpointName <name>: Create snapshotRevert-HoloDeckCheckpoint -CheckpointName <name>: Revert to snapshot
Utilities
.\Reset-Lab.ps1 -TargetSnapshot <name>: Full lab reset.\Show-LabStatus.ps1: Display current healthTest-HoloDeckConfig -ConfigFile <path>: Validate config before deployment
Scheduling
Schedule weekly snapshots via Windows Task Scheduler:
Action: PowerShell -ExecutionPolicy Bypass -File C:\Holodeck\holodeck-runtime\ScheduledSnapshot.ps1
Trigger: Weekly on Friday at 18:00
Idempotency
All scripts are designed to be re-runnable. If a step fails, you can run again without side effects.
' | Out-File C:\Holodeck\holodeck-runtime\AUTOMATION_README.md -Encoding UTF8
Validation Gate
Check: Verify: (1) HoloDeckOps.psm1 module exists with at least 3 functions, (2) Reset-Lab.ps1 script created and readable, (3) ScheduledSnapshot.ps1 created, (4) Config validation module created, (5) Status dashboard script created, (6) Module functions tested successfully, (7) Automation README documented
Expected: Complete automation framework in place for repeatable, idempotent lab operations
Common Errors
Final Validation
The Holodeck instance lifecycle is fully managed: snapshots checkpoint progress, destroy-redeploy cycles validate reproducibility, config templates enable multi-version testing, side-by-side instances demonstrate version differences, and automation scripts enable repeatable operations. You have operational maturity equivalent to production systems — the skills transfer directly.
✓ Snapshots: At least 2 named checkpoints per VM with multi-checkpoint strategy documented → Naming convention follows <component>-<phase>-<version> pattern
✓ Destroy-redeploy: Instance removed completely + redeployed from config, times measured → Redeploy time ~2-3x slower than snapshot revert; datastore space reclaimed
✓ Config management: Base + deployed + overlay configs organized in version-controlled structure → At least 2 deployed configs with naming convention and history log
✓ Version comparison: VCF 5.2.x and 9.0.x deployed, 5+ differences documented → VCDX-relevant architectural insights written (UI, components, licensing)
✓ Automation: 5+ PowerShell functions + reset script + validation + status dashboard → All scripts tested and runnable; module imports successfully
Cleanup / Restore
Snapshot: holodeck-07-complete
• Take final snapshot after all tasks complete: Get-VM -Name 'Holo-*' | New-Snapshot -Name 'holodeck-07-complete' -Description 'Post lifecycle management lab' -Memory:$false -Quiesce:$false
• Consolidate intermediate snapshots to reduce datastore overhead (keep only holodeck-02-complete and holodeck-07-complete): Get-Snapshot -Name 'lcm-*' | Remove-Snapshot -Confirm:$false
• Archive config files: Copy-Item .\configs\deployed\* .\configs\base\archive\
• Back up automation scripts to external location for future reference
Design Reflection (VCDX)
A VCDX panelist examining your operational maturity would probe: How do you manage multiple lab scenarios simultaneously? What happens when a deployment fails mid-way — can you roll back? How would you standardize Holodeck deployments across a team? Can you articulate the trade-offs between snapshots (fast, state-dependent) and redeploy (slower, reproducible)? How do version-specific differences affect your design when planning multi-version testing? This lab demonstrates that you think operationally — not just about deployment, but about repeatability, auditability, and automation. That's what separates architects from operators.
Requirements
- Support rapid environment reset for iterative design testing
- Enable concurrent testing of multiple VCF versions on shared infrastructure
- Provide reproducible, version-controlled configurations for auditing and compliance
- Detect and cleanup orphaned resources to prevent resource exhaustion
- Automate routine operations (snapshot, reset, redeploy) to reduce human error
Constraints
- Snapshots are I/O and storage-intensive — only practical for lab, not production
- Concurrent instances severely constrain physical host resources (512 GB RAM limits to 1-2 full instances)
- Nested virtualization overhead means performance characteristics don't reflect production
- Config templating requires discipline — unused configs accumulate and complicate management
Assumptions
- Holodeck Toolkit is the source of truth for instance lifecycle (Prepare, Start, Remove)
- PowerShell is the automation layer (vs REST API, which is limited)
- Snapshots are non-memory to reduce I/O overhead and enable faster revert
- Configs are version-controlled and organized (not scattered across multiple directories)
Risks
- Snapshot chain corruption causes restore failures — IMPACT: environment loss, MITIGATION: periodic consolidation, test restores regularly
- Config drift (local edits not committed) causes reproducibility failures — IMPACT: deployment fails in CI/CD, MITIGATION: enforce config validation before deploy
- Resource exhaustion during multi-instance testing cascades to physical host failure — IMPACT: total lab loss, MITIGATION: monitor resource usage, implement early warning alerts
- VCF version incompatibilities (e.g., NSX bundle mismatch) cause silent failures — IMPACT: incomplete deployments, MITIGATION: automated manifest validation, test content packs before production use
Self-Assessment Discussion Prompts
- When would you snapshot vs redeploy? What are the trade-offs in terms of time, disk space, and accuracy?
- You have 512 GB RAM and need to test VCF 9.0.2, VCF 5.2.1, and a future VCF release. How would you manage the environment?
- Snapshot consolidation is manual and labor-intensive. How would you automate it in a production lab environment?
- A junior team member misses a step in the reset-lab script and leaves the environment in an inconsistent state. How would you prevent this?
- How do version-specific config differences (e.g., NSX 9.0.x vs NSX 4.x) affect your testing methodology?
- If you needed to provision Holodeck labs for 10 VCDX candidates simultaneously, what changes to your automation strategy would be needed?
Extensions
Build a Holodeck-as-Code (HaC) Framework
Extend the config templating into a full Infrastructure-as-Code pattern. Use Terraform or Ansible to orchestrate Holodeck provisioning. This transforms lab setup from manual scripts to declarative configuration — a production-grade capability.
much harderImplement Automated Health Checks and Alerting
Build a monitoring dashboard that continuously checks Holodeck instance health (SDDC Manager availability, NSX fabric status, datastore space). Trigger automated remediation (snapshot revert, restart failed VMs) on detected anomalies.
harderCreate a Multi-Tenant Lab Hosting Service
Design a system where multiple users can provision independent Holodeck instances on shared infrastructure. This requires quota management, RBAC, automated cleanup of abandoned instances, and cost tracking. A VCDX-level challenge.
much harderIntegrate CI/CD Pipeline for Automated Lab Testing
Build a GitOps workflow where pushing a config to a Git repo automatically triggers Holodeck deployment, runs validation tests, and reports results. This demonstrates integration of lab infrastructure into development workflows.
harderDeploy Holodeck Across Multiple Physical Hosts
Extend the framework to span multiple physical ESXi hosts (e.g., two separate lab pods). Implement cross-site networking, DNS resolution, and coordinated lifecycle management. This is production-equivalent complexity.
much harder⚠ Known Pitfalls (from Community KB)
References
- VMware Cloud Foundation Deployment and Upgrade — Lifecycle ManagementTier 1 — Official
Official Broadcom docs covering production VCF lifecycle — concepts apply directly to Holodeck lab management - Holodeck Toolkit — GitHub Issues & DiscussionsTier 1 — Official
Open issue tracker with real-world failure modes and community solutions. Thread 51 (datastore selection), Thread 37 (FRR service), Thread 0 (ESXi sizing) are directly relevant. - PowerShell Desired State Configuration (DSC) for Infrastructure AutomationTier 2 — VMware Press
Framework for idempotent automation. Patterns apply to Holodeck snapshot scheduling and remediation. - VCF Planning and Preparation Workbook — Template & ChecklistTier 1 — Official
Production design checklist — repurpose for lab configuration planning (Task 3) - William Lam — VCF Holodeck Tips & TricksTier 3 — Expert Blog
Community expertise on Holodeck optimization, snapshot strategies, and multi-instance management