Lab: VM Snapshot Consolidation & Lock File Cleanup
Objectives
- Lab: VM Snapshot Consolidation & Lock File Cleanup
Prerequisites
VCF lab environment deployed and operational
Lab Environment
Standard VCF lab environment for vSphere Foundation 9.0 Support (Support Specialist)
Tasks
Task 1 Lab: VM Snapshot Consolidation & Lock File Cleanup
Snapshot consolidation and lock file cleanup are common Day-2 support tasks. Understanding when snapshots need consolidation (parent/child disk chain mismatch) and how lock files prevent VM operations is essential for resolving stuck VMs.
Step 1
Create VM with multiple snapshots: Power on VM → Take 5 snapshots (VM → Snapshot → Take Snapshot, add name)
Step 2
Monitor: vCenter → VM → Edit Settings → Note snapshot chain depth = 5
Step 3
Delete oldest snapshot: vCenter → VM → Snapshot → Delete Snapshot → Wait for consolidation (watch Recent Tasks)
Step 4
Check storage impact: vCenter → Datastore → Details tab shows freed space after consolidation
Step 5
Test lock file recovery: SSH to ESXi → find /vmfs/volumes -name "*.vmx.lck" → Identify lock for test VM
Step 6
Power off VM → Delete lock file → Power on VM (verify no "file already in use" errors)
Validation Gate
Check: Identify VMs with snapshots >24h old, consolidate snapshot chains, verify no lock files remain, and confirm VM operations (power on/off, vMotion) succeed
Expected: No VMs with snapshots >24h (or documented exceptions). Consolidation successful (no delta disks outside snapshot manager). No orphaned lock files. VM operations complete without errors.
Common Errors
Deleting snapshot files manually instead of using vCenter consolidation
Fix: Never delete .vmdk delta files manually from the datastore. Use vCenter: VM → Snapshot → Consolidate. Manual deletion breaks the disk chain and can cause data loss. If consolidation fails, collect vm-support bundle and engage Broadcom support.
Not checking for lock files when a VM fails to power on or migrate
Fix: Lock files (.lck directories/files) indicate another process has claimed the VM's files. Common causes: previous host still holds lock after HA event, backup software holding snapshot lock, or orphaned lock from crashed hostd. Check: ls -la /vmfs/volumes/<datastore>/<VM>/ for .lck files. Remove only if you've confirmed no other process is using the VM.
Forcing VM power-off without checking for active snapshots
Fix: Force-killing a VM during an active snapshot operation (create, delete, consolidate) can leave the snapshot chain in an inconsistent state. Check snapshot status before force operations. If the VM is stuck in a snapshot operation, wait for the task to complete or timeout before forcing.
Not monitoring snapshot age as a proactive practice
Fix: VMs running on old snapshots (>72 hours) accumulate delta disks that degrade performance exponentially. Configure vCenter alarms for snapshot age >24 hours. Common culprits: backup software that creates snapshots but doesn't clean them up, and manual snapshots forgotten after patching.
Final Validation
Lab completed successfully
✓ All steps completed → No errors observed
Cleanup / Restore
• Revert to snapshot if needed
Design Reflection (VCDX)
Snapshot management is an operational hygiene topic in VCDX. Designs should include snapshot policy (max age, monitoring, automated cleanup) as part of the operations plan.
Requirements
- Monitor snapshot age across all VMs
- Consolidate snapshot chains when needed
- Resolve lock file issues for stuck VMs
Constraints
- Never manually delete snapshot delta files
- Lock files must be verified as orphaned before removal
- Consolidation requires sufficient datastore free space
Assumptions
- Backup software is configured to clean up snapshots after backup
- vCenter alarms are set for snapshot age monitoring
Risks
- Performance degradation from long-running snapshots
- Data loss from manual snapshot file deletion
⚠ Known Pitfalls (from Community KB)
Deleting .vmdk delta files directly from the datastore — this breaks the disk chain. Always use vCenter Consolidate.
Assuming all .lck files are orphaned — verify no backup or HA process is actively using the VM before removing locks.