VCP-VCF 9.0 Architect (2V0-13.25)
Comprehensive study guide for VCP-VCF 9.0 Architect.
Exam Blueprint Weights
Version Evolution
VCF architect skills evolved from vSphere-centric design (VCF 3.x) to multi-domain architecture (5.x) to unified platform design (9.0). Key shifts: NSX integration changed network architecture fundamentals, vSAN became the default storage model, and VCF Operations replaced multiple monitoring tools. Architect exam now covers workload domain design, management domain sizing, and cross-domain networking.
Learning Outcomes
- VCDX Insight: During design review, candidates must articulate how each design decision traces back to a specific requirement. Avoid over-engineering (e.g., building 99.999% availability when 99.9% is the stated requirement) and be ready to show the cost of each extra nine.
- Design Pattern: For stretched cluster across two datacenters, place witness at a third location (cloud, co-location facility, even a small edge appliance). If no third location exists, a stretched cluster is not the right pattern: a witness co-located with one data site cannot arbitrate that site's failure, so choose active-passive replication instead.
- DFW Rule Performance: More rules = higher CPU overhead. Consolidate rules using groups and tags. Aim for <100 rules for typical environment. Use rule hit counters to identify unused rules monthly.
- Compliance Audit Preparation: Maintain a control matrix (Requirement ID → Design Component → Evidence) updated quarterly. Assign ownership (e.g., Security team = DFW rules, Ops team = backup verification) so every control has a named owner when the auditor asks.
- VCDX Insight: A complete design decision register is non-negotiable. Panelists expect you to explain "why this shape, not that shape?" for every choice. Document assumptions (e.g., "CPU contention acceptable up to 15% %RDY for tier-2 workloads") so the panel can test them rather than guess at them.
- Obj 1.1/3.1: Gather business objectives, differentiate business vs technical requirements, create requirements traceability matrix
- Obj 2.1: Evaluate VCF architecture options — standard, consolidated, multi-site, multi-AZ — and justify selection
- Obj 3.10/3.11: Design workload migration strategy (wave planning, HCX), VCF Automation tenant model, self-service governance, and modern application consumption
VCF Design Methodology — Objectives 1.1 through 1.7#
The VCF Architect exam Section 1 tests design methodology fundamentals. Every objective here applies across all VCF design decisions.
Objective 1.1 — Differentiate Business and Technical Requirements
Business requirements come from stakeholders and define WHAT the solution must achieve: RTO/RPO targets (e.g., RTO < 1 hour, RPO < 15 minutes), SLA uptime commitments (99.9% = 8.76 hours downtime/year, 99.99% = 52.6 minutes/year, 99.999% = 5.26 minutes/year), budget constraints, compliance mandates (HIPAA, PCI-DSS, GDPR, SOC2), geographic distribution needs, growth projections (3-5 year capacity plan), and time-to-delivery expectations.
Technical requirements translate business needs into HOW: workload characterization (CPU overcommit ratios, memory reservation policies, storage IOPS profiles), VM density targets, network segmentation model, storage tier performance (vSAN ESA NVMe latency < 200 microseconds vs HDD-based OSA), encryption requirements (vSAN data-at-rest encryption, NSX in-transit encryption), identity integration (Active Directory, LDAP, SAML), and automation maturity level.
Requirements Traceability: Every technical requirement must trace to a business requirement. Example: Business says "zero data loss for financial transactions" → Technical translates to "synchronous replication with RPO=0, requiring vSAN stretched cluster with < 5ms RTT between sites." A requirements traceability matrix (RTM) maps each business requirement to one or more technical requirements, design decisions, and validation tests.
Objective 1.2 — Differentiate Conceptual, Logical, and Physical Design
Conceptual Design: High-level architecture answering "what are we building?" — defines scope, boundaries, and major components without specifying technology. Example: "Two-site active-active private cloud with self-service consumption and automated disaster recovery." Audience: business stakeholders and CIO. Contains no vendor-specific technology references.
Logical Design: Technology-specific architecture answering "how will we build it?" — defines VCF components, relationships, network topology, security zones, storage tiers, and automation workflows WITHOUT specifying exact hardware models, IP addresses, or host counts. Example: "VCF management domain with 4 ESXi hosts, vSAN ESA all-NVMe storage, NSX overlay networking with Tier-0/Tier-1 gateway hierarchy, VCF Operations for monitoring." Audience: solution architects and technical leads.
Physical Design: Implementation-ready specification answering "what exactly do we deploy?" — defines specific server models (Dell VxRail E660F), NIC configurations (dual 25GbE for management, dual 100GbE for vSAN/vMotion), IP addressing schemes, VLAN assignments, rack layouts, power/cooling calculations, and cable maps. Example: "4x Dell PowerEdge R760, each with 2x Intel Xeon 6430 (32 cores), 1TB RAM, 8x 3.84TB NVMe TLC, 2x Mellanox ConnectX-6 100GbE." Audience: implementation engineers and procurement.
Key Distinction: Logical design is technology-specific but environment-independent (portable across sites). Physical design is environment-specific (tied to a particular datacenter, rack, and IP scheme).
Objective 1.3 — Differentiate Requirements, Constraints, Assumptions, Risks (RCAR)
Requirements: What the solution MUST do. Non-negotiable functional and non-functional needs. Example: "Must support 500 VMs with 99.99% availability."
Constraints: Limitations that CANNOT be changed. External boundaries the design must work within. Example: "Maximum budget of $2M," "Cannot add a third datacenter site," "Must use existing Cisco Nexus 9000 switches," "Only 8 rack units available."
Assumptions: Beliefs about the environment that are EXPECTED to be true but not guaranteed. Must be validated during implementation. Example: "Network latency between sites is < 5ms RTT," "All ESXi hosts will have identical hardware," "10GbE network available for vMotion." CRITICAL: Every assumption is a potential risk. If an assumption proves false, the design may fail.
Risks: Potential problems that COULD impact the design. Each risk needs probability, impact, and mitigation. Example: "WAN link congestion could increase latency beyond 5ms (probability: medium, impact: high). Mitigation: dedicated MPLS circuit with QoS for vSAN traffic, monitoring with threshold alerts."
VCDX Defense: Panelists will test whether you can distinguish these four categories. A common trap: stating "we assume the budget is $2M" — budget is a CONSTRAINT (fixed), not an assumption (uncertain). Another trap: "risk is that we run out of storage" — this is too vague. State the specific condition, probability, impact, and mitigation.
Objective 1.4 — Differentiate AMPRS Design Qualities
Every design decision impacts one or more of the five design qualities. The architect must understand trade-offs between them.
Availability: Can the system survive component failures? Measured in uptime percentage. Key VCF mechanisms: vSphere HA (host failure), vSAN data redundancy (disk/host failure), NSX Edge HA (gateway failure), stretched cluster (site failure). Trade-off: higher availability = more redundant components = higher cost and complexity.
Manageability: Can the system be operated, maintained, and evolved efficiently? Key VCF mechanisms: SDDC Manager lifecycle management, VCF Operations monitoring, VCF Automation self-service, fleet management. Trade-off: more automation = less operational burden but higher initial setup complexity.
Performance: Does the system meet latency, throughput, and IOPS requirements? Key VCF mechanisms: vSAN ESA single-tier NVMe (< 200 microsecond latency), NSX distributed routing (East-West without hairpinning), DRS load balancing. Trade-off: higher performance = more expensive hardware, potentially less consolidation.
Recoverability: Can the system recover from failures and disasters? Measured in RTO/RPO. Key VCF mechanisms: vSAN native replication, VMware Live Site Recovery (formerly SRM) orchestrated failover, vSphere Replication, backup integration (Veeam, Cohesity). Trade-off: lower RTO/RPO = synchronous replication = higher bandwidth costs and latency sensitivity.
Security: Is the system protected against threats? Key VCF mechanisms: NSX DFW micro-segmentation, vSAN encryption (at-rest and in-transit), certificate management, Identity Broker SSO, CIS compliance benchmarks. Trade-off: stronger security = more operational overhead (certificate rotation, rule management, audit logging).
VCDX Principle: No design quality exists in isolation. Improving availability (stretched cluster) impacts performance (cross-site latency), cost (duplicate infrastructure), and manageability (more complex operations). The architect must articulate these trade-offs explicitly.
Objective 1.5 — Develop and Document Risk Mitigation Strategy
Risk Register Structure: For each identified risk, document: Risk ID, Description, Category (technical/operational/financial/compliance), Probability (High/Medium/Low), Impact (High/Medium/Low), Risk Score (Probability x Impact), Mitigation Strategy, Residual Risk (after mitigation), Owner, Review Date.
Mitigation Strategies: Accept (risk is low and cost of mitigation exceeds impact), Mitigate (reduce probability or impact), Transfer (insurance, SLA penalties, vendor responsibility), Avoid (change design to eliminate the risk entirely).
VCF-Specific Risk Examples:
- Single management domain failure takes down all fleet management → Mitigate: vSphere HA for the vCenter VM (vCenter HA is not supported in VCF, Table 670), management domain on dedicated hardware, regular backups, documented manual recovery procedure.
- vSAN rebuild storm after multiple disk failures → Mitigate: configure rebuild rate throttling, maintain 25% free capacity buffer, proactive disk health monitoring via VCF Operations.
- NSX control plane split-brain during network partition → Mitigate: NSX Manager cluster across fault domains, anti-affinity rules, network path redundancy.
Objective 1.6 — Document Design Decisions
Every design choice must be formally documented with this structure:
- Decision ID: Unique identifier for traceability (e.g., ARCH-COMPUTE-001)
- Decision: Specific technical choice made
- Justification: Why this choice was selected (trace to requirement)
- Implication: Consequences of this choice (positive and negative)
- Alternative Considered: Other options evaluated and why rejected
- Risk: What could go wrong with this choice
- Dependency: Other decisions this depends on or enables
Example:
Decision ID: ARCH-STORAGE-003
Decision: Deploy vSAN ESA with RAID-5 erasure coding (4+1) instead of RAID-1 mirroring
Justification: Business requirement BR-007 specifies 80% storage utilization efficiency. RAID-5 provides 80% efficiency vs RAID-1 at 50%.
Implication: vSAN ESA builds RAID-5 as 4+1 only on clusters of 6 or more hosts; below 6 hosts it uses 2+1 (about 67% efficiency), which fails BR-007 — so the cluster minimum becomes 6 hosts. Write performance slightly lower than RAID-1. Single host failure tolerated without data loss.
Alternative: RAID-1 mirroring (rejected: only 50% efficiency, doesn't meet BR-007); RAID-6 (4+2) (rejected: 67% efficiency does not meet BR-007; same 6-host minimum)
Risk: RAID-5 rebuild on large NVMe drives takes longer than RAID-1 resync. If a second failure occurs during rebuild, data loss is possible. Mitigation: maintain hot spare capacity, monitor rebuild progress.
Dependency: Depends on ARCH-COMPUTE-001 (minimum 6 hosts per cluster, so that ESA builds the 4+1 layout)
Objective 1.7 — Develop a Design Validation Strategy
A design validation strategy ensures every requirement is testable and every design decision is verifiable before production deployment.
Validation Categories:
- Functional Validation: Does the design meet requirements? Test: deploy workloads matching the design spec, verify SLA compliance, measure actual RTO/RPO via failover testing.
- Performance Validation: Does the design meet performance targets? Test: run HCIBench or vdbench with production-representative I/O profiles, measure latency/IOPS/throughput under load.
- Availability Validation: Does the design survive failures? Test: simulate host failure (HA failover), disk failure (vSAN rebuild), site failure (stretched cluster switchover), NSX edge failure (gateway HA).
- Security Validation: Does the design meet security requirements? Test: run CIS benchmark compliance scan, verify micro-segmentation rules block unauthorized traffic via Traceflow, test certificate rotation procedure.
- Operational Validation: Can the design be operated? Test: perform lifecycle upgrade in lab, execute backup/restore procedure, verify monitoring alerts fire correctly.
Validation Traceability: Each validation test maps to a specific requirement and design decision. The traceability chain: Business Requirement → Technical Requirement → Design Decision → Validation Test → Pass/Fail Result.
Holodeck Lab Strategy: Use Holodeck Toolkit to build a representative VCF environment and execute all validation tests before production deployment. This is exactly what VCDX candidates should do — validate design decisions with evidence from a working lab.
Key Takeaways
- Obj 1.1: Business requirements define WHAT (SLA, budget, compliance); technical requirements define HOW (CPU, storage, network specs). Every technical requirement must trace back to a business requirement.
- Obj 1.2: Conceptual = scope/boundaries (no technology), Logical = technology-specific but environment-independent, Physical = implementation-ready with exact hardware/IP/VLAN specs.
- Obj 1.3: RCAR — Requirements are non-negotiable, Constraints are fixed boundaries, Assumptions are uncertain beliefs (each is a potential risk), Risks need probability + impact + mitigation.
- Obj 1.4: AMPRS qualities (Availability, Manageability, Performance, Recoverability, Security) are interconnected — improving one often impacts others. The architect must articulate trade-offs.
- Obj 1.5-1.7: Risk register with Accept/Mitigate/Transfer/Avoid strategies. Design decisions documented with ID/Decision/Justification/Implication/Alternative/Risk. Validation strategy maps tests to requirements.
Conceptual Model Development — Objective 3.2#
A conceptual model translates gathered business objectives into a high-level architecture vision. It is technology-agnostic and stakeholder-facing — no vendor names, no product versions, no IP addresses.
Purpose of the Conceptual Model
The conceptual model answers three questions: (1) What business capabilities does the solution provide? (2) What are the major architectural boundaries and relationships? (3) What quality attributes (AMPRS) does the design prioritize?
Building a VCF Conceptual Model — Step by Step
Step 1 — Define Business Capabilities: List the capabilities the private cloud must deliver. Example capabilities: "Self-service VM provisioning for 10 development teams," "Automated disaster recovery with < 1 hour RTO," "Centralized compliance monitoring across all workloads," "Network isolation between business units."
Step 2 — Identify Architectural Boundaries: Define the major zones/domains: Management zone (platform control plane), Workload zones (tenant compute), Edge zone (external connectivity), Operations zone (monitoring/logging). These become the logical separation points in the design.
Step 3 — Map Quality Attributes to Capabilities: For each capability, identify which AMPRS quality is primary. Self-service provisioning → Manageability. Disaster recovery → Recoverability. Compliance monitoring → Security. Network isolation → Security + Availability.
Step 4 — Define External Interfaces: Identify integration points: corporate Active Directory, external DNS, NTP, syslog/SIEM, backup infrastructure, WAN/internet connectivity, cloud provider interconnects. These become the physical design boundary conditions.
Step 5 — Document Constraints and Assumptions: Capture all constraints (budget, site limitations, existing hardware reuse) and assumptions (network bandwidth, vendor support availability, team skill level) that will shape the logical and physical designs.
Conceptual Model Artifacts
The conceptual model produces three artifacts:
- Conceptual Architecture Diagram: Boxes-and-lines showing major zones, relationships, and external interfaces. No technology names — just functional labels ("Compute Platform," "Network Fabric," "Monitoring System").
- Quality Attribute Priority Matrix: Ranked list of AMPRS qualities with justification. Example: "Security > Availability > Performance > Manageability > Recoverability" with rationale for the ordering.
- Constraints and Assumptions Register: Formal list of all constraints and assumptions with ownership and validation plan.
VCDX Defense Application: The conceptual model is your first slide in a VCDX defense. Panelists want to see that you understood the business problem BEFORE jumping to technology. A common failure: presenting a physical design (server models, IP addresses) without first establishing WHY those choices were made. The conceptual model provides the WHY.
Key Takeaways
- Obj 3.2: Conceptual model is technology-agnostic — no vendor names, no IP addresses, no product versions. It maps business capabilities to architectural boundaries.
- Three artifacts: Conceptual Architecture Diagram (zones/relationships), Quality Attribute Priority Matrix (AMPRS ranking), Constraints/Assumptions Register.
- VCDX Defense: The conceptual model is your opening argument — it establishes WHY before WHAT. Panelists will ask 'what business problem does this solve?' before examining any technical detail.
VCF Logical Design Decisions — Objective 3.3 (All Sub-Objectives)#
Logical design translates the conceptual model into technology-specific architecture. It defines WHAT VCF components are used and HOW they relate — without specifying exact hardware, IP addresses, or host counts. The 2V0-13.25 blueprint tests logical design for 8 VCF areas.
Obj 3.3.1 — Logical Design: VCF Prerequisites
DNS Architecture: Forward and reverse DNS for all VCF management components (vCenter, NSX Manager, SDDC Manager, ESXi hosts). Dedicated DNS zone (e.g., vcf.lab.local) recommended. DNS must be available BEFORE VCF bring-up — circular dependency if DNS runs on VCF itself.
NTP Architecture: All VCF components must synchronize to the same NTP source. Maximum acceptable drift: 5 seconds (vSAN CMMDS requirement). Stratum-2 or better NTP servers, minimum 2 for redundancy.
Network Prerequisites: Management network (routable, all components), vMotion network (dedicated, jumbo frames optional), vSAN network (dedicated VLAN; one consistent MTU for all vSAN traffic, 9000 recommended), NSX overlay network (host and edge TEP VLAN: minimum MTU 1700, Geneve functions from 1600, 9000 recommended — VCF 9.0 Table 695), uplink networks for north-south traffic.
Certificate Authority: VCF supports Microsoft CA, OpenSSL CA, or VMCA as intermediate CA. Decision: use enterprise CA for production (compliance), VMCA for lab/dev. Certificate lifecycle: auto-renewal for internal certs, manual process for external-facing certs.
Active Directory/LDAP: Identity source for SSO. VCF Single Sign-On (VCF Identity Broker, embedded or appliance mode) provides SSO across vCenter, NSX, VCF Operations, VCF Automation and the other VCF Operations components; SDDC Manager and ESX are excluded and keep their own authentication (VCF 9.0, Configuring VCF Single Sign-On). Decision: dedicated service account per VCF component vs shared service account (dedicated is more secure, shared is simpler).
Obj 3.3.2 — Logical Design: Fleet Topologies
Single-Instance Fleet: One SDDC Manager managing one management domain and multiple workload domains. Suitable for: single datacenter, < 1000 VMs, single operations team. Simplest to deploy and manage.
Multi-Instance Fleet: Multiple SDDC Manager instances, each managing its own VCF instance. Suitable for: multi-region deployments, organizational isolation requirements, separate management domains per business unit. VCF Fleet Management provides single-pane visibility across instances.
Consolidated Architecture: Management and workload VMs share the same cluster (minimum 4 hosts). Suitable for: small deployments, edge/ROBO locations, cost-constrained environments. Trade-off: management workload contention possible, but reduced hardware footprint.
Standard Architecture: Dedicated management domain cluster (4 hosts) separate from workload domain clusters. Suitable for: production enterprise deployments, compliance requirements demanding management isolation. Trade-off: more hardware required, but clean separation of concerns.
Multi-AZ (Availability Zone): Stretched clusters spanning two sites with witness at a third site. Each site is an AZ. Suitable for: active-active datacenter designs, metro-distance deployments (< 5ms RTT (no geographic distance restriction — only latency matters)). Trade-off: 2x infrastructure cost, complex networking, WAN dependency.
Obj 3.3.3 — Logical Design: Network Infrastructure
Underlay Network: Physical switching fabric that carries all VCF traffic types. Design decisions: L2 vs L3 spine-leaf topology (L3 preferred for scale), VLAN allocation strategy (separate VLANs per traffic type), MTU 9000 for vSAN and NSX overlay, redundant uplinks with LACP or active-standby.
Overlay Network: NSX GENEVE-encapsulated virtual networks running on top of the physical underlay. Design decisions: transport zone scope (per-cluster vs shared across clusters), TEP IP pool sizing, host TEP vs edge TEP separation.
Traffic Flow Design: Define North-South (external to/from VCF) and East-West (VM-to-VM within VCF) traffic patterns. North-South flows through NSX Tier-0 gateway → physical router. East-West stays on NSX distributed router (DFW enforces segmentation). Design decision: centralized vs distributed services (NAT, load balancing, firewall).
Obj 3.3.4 — Logical Design: Management Domain
Management domain hosts the VCF control plane: vCenter Server, NSX Manager cluster (3 nodes), SDDC Manager, VCF Operations, VCF Automation (optional). Logical design decisions:
Cluster sizing: Minimum 4 hosts for vSAN (FTT=1 with RAID-5). Recommended: 4 hosts for small (< 500 workload VMs), 6-8 hosts for large (> 1000 workload VMs) to handle management workload growth.
Resource reservation: Management VMs should have CPU and memory reservations to prevent workload contention from affecting control plane. vCenter: reserve what its deployment size needs (for example Medium, 8 vCPU and 30 GB for up to 400 hosts or 4,000 VMs; vSphere 9.0 docs p.309). NSX Manager: 6 vCPU, 24GB reserved per node. SDDC Manager: 4 vCPU, 16GB reserved.
Anti-affinity: NSX Manager nodes must run on separate hosts (anti-affinity rules). The vCenter VM is protected by vSphere HA with high restart priority (vCenter HA active/passive/witness is not supported in VCF, Table 670).
Storage policy: Management domain vSAN policy: FTT=1, RAID-5 erasure coding for space efficiency (management VMs are not IOPS-intensive). Alternatively FTT=1, RAID-1 for simpler operations.
Obj 3.3.5 — Logical Design: Workload Domain
Workload domains host tenant/application VMs. Each workload domain has its own vCenter instance (since VCF 5.0+) and dedicated clusters. Logical design decisions:
Domain boundary strategy: One workload domain per business unit (strong isolation), one per environment (dev/staging/prod), or one per application tier (web/app/DB). Decision depends on: security isolation requirements, lifecycle independence, operational team structure.
Cluster composition: All hosts in a cluster must be identical (CPU, memory, storage, NIC). Mixed clusters cause DRS imbalance and vSAN degradation. Design decision: cluster size (minimum 4 for vSAN, recommended 8-16 for production).
Storage tier: vSAN ESA (NVMe-only, single-tier, higher performance) vs vSAN OSA (disk groups of one flash cache device plus HDD or flash capacity devices, lower cost; VCF 9.0.1 docs p.1653). Decision based on workload IOPS requirements and budget. Can mix storage tiers across different workload domains.
Network isolation: Each workload domain can have independent NSX transport zones for network isolation, or share transport zones for cross-domain communication. Decision depends on security requirements.
Obj 3.3.6 — Logical Design: VCF Networking (NSX)
NSX Manager cluster: 3-node cluster for production (quorum-based). The management domain has its own NSX Manager cluster; workload domains cannot use it (VCF 9.0 Table 2, Types of VCF Domains). The first workload domain deploys a new NSX Manager cluster into the management domain; later workload domains either share an existing workload NSX Manager cluster or get a dedicated one. Design decision: shared = fewer appliances and one policy scope; dedicated = isolation and independent lifecycle. Further decision: single vs multi-site NSX Federation (multi-site for cross-datacenter consistent policy).
Tier-0 Gateway: Provides north-south connectivity. Design decisions: active-active (ECMP) for high throughput or active-standby for simplicity. BGP peering with physical routers. Route redistribution strategy (which routes advertise to physical network).
Tier-1 Gateways: Provides routing for tenant segments. Design decisions: one Tier-1 per workload domain (simple), one per application (granular control), or one per tenant (multi-tenancy). Tier-1 connects upstream to Tier-0 for external access.
DFW Micro-segmentation: Logical security policy model. Design decisions: rule category strategy (Emergency/Infrastructure/Environment/Application/Default), Applied To scope (limit rule processing per VM), tag-based grouping (application-centric) vs IP-based grouping (network-centric).
Edge Cluster: Hosts Tier-0 and Tier-1 service routers. Design decisions: dedicated edge cluster (recommended) vs shared with compute. Edge VM sizing: Large (8 vCPU, 32GB) for production. Minimum 2 Edge VMs for HA.
Obj 3.3.7 — Logical Design: VCF Automation
Deployment model: single node, or a three-node cluster for high availability (VCF 9.0 docs p.485). Decision based on availability requirements.
Tenancy model: the provider administrator creates Organizations (tenants or lines of business) and adds workload domains to them. All Apps Organizations use workload domains with Supervisor clusters that run both containers and VMs; VM Apps Organizations use workload domains whose clusters run only VMs, and one workload domain cannot serve both types (VCF 9.0 docs p.467).
Project model: Organization Administrators represent application teams as Projects (a default Project is created with each Organization) and assign resource envelopes to Projects as vSphere Namespaces, which set CPU and memory limits and reservations and storage classes (VCF 9.0 docs p.101).
Content strategy: Content Libraries created by the provider can be shared with all Organizations, and Blueprints and Orchestration workflows can be shared too (p.101). The Assembler, Service Broker and Orchestrator interfaces are merged into one UI: Consume hosts the Catalog and Deployments, Design hosts Blueprints (p.104).
Obj 3.3.8 — Logical Design: VCF Operations
Deployment model (VCF 9.0.1 docs p.859): Simple model — a single analytics node for lab or small estates; High Availability model — a three-node analytics cluster plus single-node VCF Operations fleet management and collector appliances. There is no two-node HA model. Scale out with data nodes and add collector nodes into collector groups for remote VCF instances.
Adapter architecture: vCenter adapter (compute/storage metrics), NSX adapter (network metrics), vSAN adapter (storage health), SDDC Manager adapter (lifecycle status). Each adapter provides specific metric domains.
Dashboard and alerting strategy: Role-based dashboards — executive summary, ops team operational view, security team compliance view. Alert definitions with notification channels (email, webhook, SNMP). Escalation hierarchy: warning → critical → emergency.
Content pack strategy: CIS compliance benchmarks, VMware hardening guide, custom VCF operational benchmarks. Decision: which compliance frameworks to enable based on regulatory requirements.
Integration with VCF Automation: VCF Operations can trigger Automation workflows based on alert conditions (e.g., auto-scale workload domain when capacity threshold reached). This creates a closed-loop operations model.
Key Takeaways
- Obj 3.3: Logical design is technology-specific (VCF, NSX, vSAN) but environment-independent (no IP addresses, no specific hardware models). It defines component relationships and architectural patterns.
- Fleet topology decision: Single-instance (simple) vs Multi-instance (isolation) vs Consolidated (cost) vs Standard (separation) vs Multi-AZ (availability). Each has clear trade-offs in cost, complexity, and availability.
- Management domain logical design: 4-8 hosts, resource reservations for control plane VMs, anti-affinity rules for the three NSX Manager nodes, vCenter protected by vSphere HA plus file-based backup (vCenter HA is not supported in VCF, Table 670), vSAN RAID-5 for space efficiency.
- Workload domain boundary strategy: per business unit (isolation), per environment (lifecycle), or per application tier (performance). Decision depends on security, operational, and compliance requirements.
- NSX logical design: Tier-0 (north-south, BGP peering) → Tier-1 (per-tenant routing) → Segments (VM connectivity). DFW provides micro-segmentation with 5 rule categories.
VCF Physical Design Decisions — Objective 3.4 (All Sub-Objectives)#
Physical design translates logical architecture into implementation-ready specifications. It defines EXACT hardware models, IP addresses, VLAN IDs, cable maps, and rack layouts. The 2V0-13.25 blueprint tests physical design for 8 VCF areas.
Obj 3.4.1 — Physical Design: VCF Prerequisites
DNS Physical Design: Specific DNS server IPs (e.g., 10.0.0.11, 10.0.0.12), zone configuration files, forward/reverse zone entries for every VCF component FQDN. DNS must be deployed and validated BEFORE VCF bring-up.
NTP Physical Design: NTP server IPs, stratum level, chrony vs ntpd configuration on ESXi hosts. Example: ntp1.corp.local (10.0.0.21, stratum-2), ntp2.corp.local (10.0.0.22, stratum-2).
Network Physical Design: Switch model and firmware (e.g., Cisco Nexus 93180YC-FX3S running NX-OS 10.3), port allocation map, VLAN table:
- VLAN 10: Management (10.10.10.0/24, MTU 1500)
- VLAN 20: vMotion (10.10.20.0/24, MTU 9000)
- VLAN 30: vSAN (10.10.30.0/24, MTU 9000)
- VLAN 40: NSX TEP (10.10.40.0/24, MTU 9000)
- VLAN 50: Uplink/External (public or routable subnet)
Cabling: VCF does not require four NICs — the Single Distributed Switch Model runs on two host NICs, or one through the API (VCF 9.0 docs p.497). This example uses 4 physical NICs to separate traffic — 2x 25GbE for management/vMotion (active-standby or LACP), 2x 25GbE or 100GbE for vSAN/NSX overlay (LACP recommended for throughput). ToR switch redundancy: dual switches per rack.
Obj 3.4.2 — Physical Design: Fleet Topology
For a standard architecture single-instance fleet:
- Management Domain: 1 rack, 4 hosts, 2 ToR switches
- Workload Domain 1: 1-2 racks, 8-16 hosts, 2 ToR switches
- Workload Domain 2: 1-2 racks, 8-16 hosts, 2 ToR switches
- Edge rack: 2 bare-metal Edge nodes (or Edge VMs on compute hosts)
Rack layout considerations: Power distribution (A+B feeds per rack), cooling (front-to-back airflow), cable management (structured cabling with patch panels), physical security (locked racks, access logging).
For multi-AZ stretched cluster:
- Site A: 4 hosts (management; stretched management default cluster needs 8 in total, VCF-VSAN-SC-REQD-CFG-006) + 3 or more hosts (workload; 6 minimum per stretched workload cluster, VCF-VSAN-SC-REQD-CFG-007) + Edge nodes
- Site B: 4 hosts (management) + 3 or more hosts (workload) + Edge nodes
- Witness Site: one vSAN witness appliance (an ESX VM) per stretched cluster, or a shared witness, in a third location outside both availability zones
- WAN: Dedicated dark fiber or MPLS circuit, < 5ms RTT, minimum 10Gbps for vSAN replication
Obj 3.4.3 — Physical Design: Network Infrastructure
Spine-Leaf Physical Topology (recommended for > 2 racks):
- 2 Spine switches (e.g., Cisco Nexus 9336C-FX2) providing inter-rack L3 routing
- 2 Leaf switches per rack (e.g., Cisco Nexus 93180YC-FX3S) providing host connectivity
- BGP EVPN/VXLAN fabric for scalable L2/L3 overlay (or simple OSPF for smaller deployments)
Port Allocation Per Host (4-NIC design):
- vmnic0 → Leaf-A port (management/vMotion uplink-1)
- vmnic1 → Leaf-B port (management/vMotion uplink-2)
- vmnic2 → Leaf-A port (vSAN/overlay uplink-1)
- vmnic3 → Leaf-B port (vSAN/overlay uplink-2)
VDS Configuration:
- Management VDS: vmnic0 + vmnic1, active-standby teaming, VLAN 10 (management) + VLAN 20 (vMotion)
- Overlay VDS: vmnic2 + vmnic3, LACP teaming, VLAN 30 (vSAN) + VLAN 40 (NSX TEP)
Physical router integration: BGP peering between NSX Tier-0 Edge and physical router. Advertised routes: workload subnets. Received routes: default route or specific external routes. BFD for fast failure detection (< 1 second).
Obj 3.4.4 — Physical Design: Management Domain
Host Specification (example for 500-VM environment):
- 4x Dell PowerEdge R760 (or HPE ProLiant DL380 Gen11, Lenovo ThinkSystem SR650 V3)
- CPU: 2x Intel Xeon Gold 6430 (32 cores each, 64 cores total per host) — meets VCF 9.0 minimum 16 cores per socket
- RAM: 1024 GB DDR5-4800 (management VMs need ~200GB, remainder for growth)
- Storage: 8x 3.84TB NVMe TLC SSD (vSAN ESA, ~30.7TB raw per host, ~123TB raw total)
- NIC: 2x Mellanox ConnectX-6 Dx dual-port 25GbE (4 ports total per host)
- Boot: 2x M.2 SATA SSD in RAID-1 (ESXi boot device)
IP Addressing (management domain):
- vCenter: 10.10.10.10
- SDDC Manager: 10.10.10.11
- NSX Manager 1/2/3: 10.10.10.12-14, VIP: 10.10.10.15
- ESXi hosts: 10.10.10.101-104
- vSAN VMkernel: 10.10.30.101-104
- vMotion VMkernel: 10.10.20.101-104
- TEP Pool: 10.10.40.101-108 (2 TEPs per host x 4 hosts, one per active overlay uplink)
VCF 9.0 Licensing: Per-core licensing model. 4 hosts x 64 cores = 256 cores. Minimum purchase: 16 cores per socket = 128 cores minimum. Actual: 256 VCF cores licensed.
Obj 3.4.5 — Physical Design: Workload Domain
Host Specification (example for performance tier):
- 8x Dell PowerEdge R760 (identical to management domain hosts for operational simplicity, or different spec for cost optimization)
- CPU: 2x Intel Xeon Gold 6438N (32 cores, optimized for VMs)
- RAM: 2048 GB DDR5 (high density for VM consolidation)
- Storage: 12x 7.68TB NVMe TLC SSD (vSAN ESA, ~92TB raw per host)
- NIC: 2x Mellanox ConnectX-6 Dx dual-port 100GbE (for high-bandwidth workloads)
Capacity Planning:
- Usable vSAN capacity: 8 hosts x 92TB raw x 0.80 (ESA RAID-5 is 4+1 at 6 or more hosts) x 0.70 (slack space) = ~412TB usable
- Compute capacity: 8 hosts x 64 cores x 4:1 overcommit = 2048 vCPUs available
- Memory capacity: 8 hosts x 2048GB x 0.85 (reservation buffer) = ~13.9TB available
Obj 3.4.6 — Physical Design: VCF Networking (NSX)
Edge Node Physical Design:
- 2x Edge VMs (Large: 8 vCPU, 32GB RAM, 200GB disk) on management domain hosts
- Or 2x bare-metal Edge nodes (dedicated hardware for high-throughput north-south traffic)
- Edge uplink: dedicated VLAN for BGP peering with physical router
NSX TEP IP Pool Sizing:
- Management domain: 4 hosts x 2 TEPs = 8 IPs + growth = /27 subnet (10.10.40.0/27)
- Workload domain 1: 8 hosts x 2 TEPs = 16 IPs + growth = /26 subnet (10.10.40.64/26)
- Edge TEPs: 2 TEPs per Edge VM x 2 edges = 4 IPs + spare = /28 (10.10.40.128/28)
(The VCF 9.0 design blueprints allocate 2 host TEP IPs per host and 2 Edge TEP IPs per Edge VM — VCF 9.0 docs p.537.)
Physical Firewall Integration: North-south traffic flow: VM → DFW → NSX Tier-1 → Tier-0 → physical firewall (Palo Alto/Fortinet) → internet. Decision: insert physical firewall inline (transparent bridge mode) or as next-hop (routed mode).
Obj 3.4.7 — Physical Design: VCF Automation
Deployment:
- Single node, or a three-node cluster for production HA (the High Availability VCF Automation model in the VCF 9.0 design blueprints); take the appliance size from the VCF 9.0 planning and sizing guidance rather than fixed VM-count thresholds
Network Requirements: Management network access to vCenter and NSX Manager APIs. All Apps Organizations need NSX Edge clusters and VPC networking in their workload domains, plus a Supervisor with NSX VPC (VCF 9.0 docs p.467).
Integration Points:
- Workload domains added to Organizations in provider management (All Apps or VM Apps)
- Identity: an identity provider configured for the Organizations (VCF Single Sign-On through VCF Identity Broker)
- VCF Operations: chargeback and showback at Organization level (p.101)
- IPAM and CMDB integrations as extensibility, designed per Organization
Obj 3.4.8 — Physical Design: VCF Operations
Appliance Sizing:
- Analytics nodes: one node (Simple model) or a three-node cluster (High Availability model); add data nodes to scale. Per-node vCPU/RAM/disk come from the VCF Operations sizing guidelines for the chosen node size (Small/Medium/Large/Extra Large) and object count — size from the sizing KB, not from a VM-count rule of thumb.
- Collectors are cloud proxies (VCF 9.0.1 docs p.2965 Table 934): Classic Small 2 vCPU / 8 GB (up to 16,000 VMs), Classic Large 4 vCPU / 32 GB (16,000–80,000 VMs), Unified Small 4 vCPU / 16 GB, Unified Large 8 vCPU / 48 GB; two or more in a collector group for HA
Storage Retention: Metric data retention affects disk sizing. Default: 6 months real-time, 3 years rolled-up. For compliance: extend to 5-7 years (increases disk requirements significantly).
Network Requirements: Collector-to-adapter traffic: port 443 (HTTPS) to vCenter, NSX Manager, SDDC Manager. Outbound: port 443 to Broadcom for content pack downloads (or offline import for air-gapped environments).
Integration with Logging: VCF Operations for Logs (formerly vRealize Log Insight): separate appliance for syslog aggregation. Sizing: 1 node per 5000 events/second. Integration: log events correlate with metric alerts for root cause analysis.
Key Takeaways
- Obj 3.4: Physical design is implementation-ready — specific server models, CPU/RAM/storage specs, IP addresses, VLAN IDs, port assignments, rack layouts, and cable maps.
- Management domain physical: 4 hosts minimum, 64+ cores per host (VCF 9.0 per-core licensing), 1TB+ RAM, 8+ NVMe drives per host for vSAN ESA, 4 NICs per host in this example (2 pairs for traffic separation; VCF also supports 2-NIC single-switch designs).
- Network physical: Spine-leaf topology for > 2 racks. Per-host 4-NIC design: 2 for management/vMotion (active-standby), 2 for vSAN/overlay (LACP). MTU 9000 for vSAN and NSX TEP VLANs.
- Capacity planning formula: Usable vSAN = hosts x raw x RAID-overhead x slack-factor. Compute = hosts x cores x overcommit-ratio. Memory = hosts x RAM x reservation-buffer.
- NSX TEP pool sizing: 2 TEPs per host (one per active overlay uplink). Size subnet to 1.5x current host count for growth. Edge TEPs on separate subnet from host TEPs.
Compute Design#
ESXi Host Sizing & Capacity Planning
Host sizing begins with workload characterization, not with purchasing decisions. Understand your VM mix before selecting hardware.
CPU Sizing Methodology:
vCPU allocation:
Sum all VM vCPU requirements + 10-15% headroom for management services + vSAN operations
Overcommit ratios:
CPU is highly virtualized; typical ratios 4:1 (bursty workloads) to 8:1 (stable baseline workloads). Advanced users understand the difference between entitlement and allocation.
pCPU selection:
Calculate required pCPUs = total vCPU demand / overcommit ratio. Example: 1000 vCPUs with 4:1 ratio = 250 pCPUs = 8 x 32-core hosts (with overhead).
Memory overcommit:
Unlike CPU, memory is not aggressively overcommitted. Guideline: 95% of host RAM allocated to VMs, with remainder reserved for hypervisor, management, ballooning overhead.
vSAN-specific overhead:
vSAN consumes host memory per disk group / storage pool and per device (OSA hosts need 32 GB for system operations, see KB 2113954; vSAN ESA requires at least 128 GB host memory; calculate the overhead with the vSAN Sizer — VCF 9.0.1 docs p.1679). Deduplication and compression save storage space within each disk group, not host RAM (p.1773) — do not count them against the memory budget.
Sizing Example:
Workload: 800 vCPUs, 3200GB VM memory, 4:1 CPU overcommit
Host Spec: 32 cores (64 logical threads), 512GB RAM
CPU Calculation:
- Required pCPUs (physical cores) = 800 / 4 = 200 cores
- Cores per host = 32 (size on cores, not hyperthreads)
- Hosts needed = 200 / 32 = 6.25 → 7 hosts
Memory Calculation:
- VM Memory = 3200GB
- Hypervisor overhead per host ≈ 12GB (kernel, drivers, management)
- vSAN overhead ≈ 16GB (disk group metadata, dedup/compression)
- Per-host allocation target = 512 - 12 - 16 = 484GB for VMs
- Memory needed = 3200 / 484 = 6.6 → 7 hosts required
Limiting factor: CPU and memory both need 7 hosts
Final design: 8-host cluster (7 + 1 for N+1 host failure)
Cluster Design & Scalability
vSphere Cluster Maximums (vSAN environment):
Maximum hosts per cluster: 64 (vSphere 7.0+) with all-flash vSAN
vSAN stretched cluster: the VCF 9.0 site-to-witness latency table covers up to 15 hosts per site (<200 ms RTT up to 10 hosts per site, <100 ms for 11–15; VCF 9.0 docs p.1617); size both sites symmetrically
Minimum hosts: 3 for a standard vSAN cluster; a two-host vSAN cluster (two data hosts plus an external witness) is a separate topology (VCF 9.0 docs p.1667)
DRS (Distributed Resource Scheduler) Configuration:
Automation level:
Manual (explicit approval required), Partially Automated (initial placement automatic; migrations shown as recommendations only), or Fully Automated (immediate migrations)
Affinity rules:
VM-to-VM affinity (keep together for low latency), VM-to-host affinity (pin to specific hardware)
Anti-affinity rules:
Separate VMs to different hosts (high availability pattern), separate VMs from host groups (licensing constraints)
VM groups:
Label VMs logically (e.g., "database tier", "web tier") for policy application
Host groups:
Designate hardware groups (e.g., GPU hosts, high-memory hosts) and apply rules to place specific workloads there
Cluster Architecture Patterns:
General-purpose cluster:
Homogeneous hardware, mixed workload types, standard sizing
Specialized clusters:
Separate clusters for CPU-intensive, memory-intensive, GPU-required, or I/O-bound workloads with right-sized hardware
Multi-tier architecture:
Separate clusters for different application tiers (presentation, application, database) enabling tier-specific policies
HA Design & Admission Control
vSphere HA protects against host failures through VM restart mechanisms. Design HA capacity carefully to ensure failover capability.
HA Admission Control Policies:
Slot-based policy:
Calculates a "slot" size from the largest CPU and memory reservations of powered-on VMs. Cluster can host N-1 failures if spare capacity ≥ 1 slot. Sensitive to large VM outliers.
Percentage-based policy:
Reserve percentage of cluster resources for failover (e.g., 25% means cluster can handle 1 failure in 4-host cluster). Simpler but less precise.
Dedicated failover hosts policy:
Explicitly reserve one host's capacity for failovers. Predictable but ties up resources; suitable for small clusters.
Disabled admission control:
Not recommended; allows VM placement that violates HA guarantees.
HA Pit:
If admission control is set to percentage-based (25%) on a 4-host cluster, 25% of CPU and memory is held in reserve across all four hosts, so only 75% of cluster capacity is usable; no host is left empty (that is the dedicated failover hosts policy). Design cluster size to balance capacity utilization and failover headroom.
Enhanced vMotion Compatibility (EVC)
EVC allows mixed-generation CPU hardware in a single cluster by masking advanced CPU features. Critical for multi-site designs or phased upgrades.
EVC Mode Selection:
Match the lowest-generation CPU in the cluster (e.g., if mixing Intel Skylake and Cascade Lake, use Skylake EVC mode)
VM Compatibility:
A running VM keeps the EVC mode it powered on with; if the cluster EVC mode is raised it does not use the new features until a full power cycle (a guest reboot is not enough)
Performance implication:
VMs lose access to newer CPU features (e.g., AVX2, TSX); measure impact for floating-point workloads
Multi-site scenario:
If replicating VMs between vSAN stretched cluster sites with different CPU generations, EVC must match across sites
Key Takeaways
- Compute density determines cluster performance: CPU overcommit ratio (typically 4:1) affects vMotion feasibility and CPU ready time.
- Host affinity and anti-affinity rules enforce compliance but must be designed for resiliency—avoid creating single points of failure.
- NUMA topology impacts performance: VMs with large vCPU counts (>16) require NUMA awareness for optimal memory latency.
Storage Design#
vSAN Architecture & Capacity Planning
vSAN is a hyperconverged storage platform that pools local storage from ESXi hosts. Architecture choices directly impact performance, capacity, and operational complexity.
- vSAN Original Storage Architecture (OSA)
- the traditional vSAN model (VCF 9.0.1 docs p.1653):
Disk groups:
Logical grouping of one flash cache device + one to seven capacity devices (VCF 9.0.1 docs p.1653). Host can have multiple disk groups for scaling.
Cache (read/write buffer):
One flash device (NVMe or SSD) per disk group acts as the write buffer (and read cache on hybrid); it is a single device, not a RAID pair — losing it fails the whole disk group.
Capacity tier:
Flash (all-flash) or HDD (hybrid) devices; single or multiple devices per disk group. The hybrid configuration is deprecated in VCF 9.0 and will be discontinued in a future release (VCF 9.0 release notes, docs p.216), so design new VCF 9 clusters all-flash.
Cache-to-capacity ratio:
Industry guideline 1:10 to 1:30. Example: 400GB NVMe cache to 4-12TB of flash capacity. If ratio is too high, cache fills unnecessarily; too low, high misses and destaging overhead.
- vSAN Express Storage Architecture (ESA)
- introduced in vSAN 8.x+:
Distributed pool architecture:
No separate cache/capacity tiers: a single all-NVMe storage pool per host that handles both caching and capacity, with no tiering: the performance and capacity legs of each object replace the OSA cache (VCF 9.0.1 docs p.1652).
Advantages:
Higher performance (no destaging bottleneck), simplified capacity planning, better dedup/compression efficiency, supports encryption natively.
Minimum requirement:
All-NVMe TLC drives (no HDDs in ESA). Minimum 1 NVMe TLC device per storage pool (official requirement). ReadyNode configurations typically require 4+ drives for production performance. Requires 128 GB minimum host memory (p.1663).
When to choose ESA:
Performance-critical workloads, cloud-native (high churn), capacity-constrained environments where dedup/compression matter.
Fault Tolerance & Capacity Math
Failures to Tolerate (FTT):
vSAN replicates objects to survive host/disk failures. FTT=1 survives 1 failure (RAID-1 two replicas, or RAID-5); FTT=2 survives 2 failures (RAID-1 three replicas, or RAID-6). Striping (stripe width) is a separate policy rule and does not add failure tolerance.
Capacity Calculation with FTT:
Object Size = 100GB (single VM disk)
FTT=1 (Mirroring - RAID 1):
Replicas = 2 (primary + 1 copy)
Effective capacity consumed = 100GB × 2 = 200GB
Usable capacity = 50% of raw
FTT=2 with Erasure Coding (RAID 6):
- Primary object = 100GB, split into 4 stripes (25GB each) + 2 parity stripes (25GB each)
- Total 6 stripes across 6 hosts = 150GB consumed
- Usable capacity = 66.7% of raw (RAID 6 = n/(n+2) for n data stripes)
- Cluster 10 hosts × 1.6TB = 16TB raw capacity
- With FTT=2 RAID 6: 16TB × 0.667 = 10.67TB usable
- With FTT=1 mirroring: 16TB × 0.5 = 8TB usable
Difference: 2.67TB (33% more usable capacity with erasure coding — but FTT=2 RAID-6 against FTT=1 RAID-1 is not like-for-like protection)
Erasure Coding vs Mirroring Trade-offs:
Mirroring (FTT=1):
Simple, low latency, 50% usable capacity, faster rebuild, better for small clusters (<6 hosts)
Erasure Coding (FTT=2):
Complex, higher latency for random writes, ~67% usable capacity, slower rebuild, better for large clusters (>6 hosts), requires even object placement across hosts
Hybrid approach:
Use mirroring for critical VMs, erasure coding for bulk storage (e.g., archives, backups)
vSAN Stretched Cluster Design
A stretched cluster spans two sites with a third witness site for quorum. Both data sites are active and every write is committed synchronously to both sites, so RPO = 0; loss of one site (or of the inter-site link) keeps the datastore available and vSphere HA restarts the affected VMs on the surviving site.
Architecture Requirements:
Network latency:
<5ms RTT between primary sites (stretched cluster); <200ms to witness (asynch commit possible). Measure with
ping -c 100 remote-site
and examine standard deviation; high variance causes problems.
Host count:
For standard stretched clusters: minimum 3 hosts per site (6 total) for quorum. 2-node stretched clusters (1 host per site + witness) are also supported for smaller deployments. Witness is deployed as a dedicated ESXi appliance (OVA). Sizing tiers: Tiny (≤750 components/≤10 VMs), Medium (≤21,833 components/≤500 VMs), Large (≤45,000 components), Extra Large (≤64,000 components). vSAN ESA does not support Tiny witness.
Partition handling:
One site is designated the preferred site. If the inter-site link fails, the witness forms quorum with the preferred site and vSAN continues there; VMs on the secondary site are restarted by vSphere HA on the preferred site. If the preferred site is isolated from both the secondary site and the witness, the witness forms quorum with the secondary site instead. Host count does not decide the outcome; the witness does (VCF 9.0, Creating a vSAN Stretched Cluster).
Witness placement:
MUST be at third location. If witness is co-located with one site, it's not truly a tiebreaker (dependent on that site's availability).
Witness VM Sizing:
Deployed using the Tiny or Medium OVA option depending on cluster size. Hosted outside the stretched cluster at the third site (a management host at a third location, a co-location or cloud host, or a shared witness host), never on the stretched cluster it arbitrates. Must use a different datastore than the vSAN stretched cluster datastore.
External Storage Integration
VCF supports FC, iSCSI, and NFS for non-vSAN deployments or hybrid configurations.
FC (Fibre Channel):
High performance (10/16/32Gbps), stable, best for database tier. Requires dedicated fabric.
iSCSI:
IP-based, easier to deploy (standard Ethernet), sufficient for most workloads. Software iSCSI on ESXi handles well; hardware iSCSI NICs reduce CPU overhead.
NFS v4.1:
Modern, supports Kerberos authentication, good for file-based workloads (VDI, file services).
Multi-site consideration:
FC requires stretching fabric (expensive); iSCSI/NFS stretch easily over IP WAN.
Datastore Sizing & Storage Policy Alignment
Datastores must map to SLA requirements through storage policies (vSAN policy objects or external storage class definitions).
Storage Tier to Policy Mapping Example:
Tier 1 - Premium (Databases, Real-time):
vSAN ESA FTT=2 RAID-6 (needs 6+ hosts), compression disabled in the policy for latency-sensitive data, all-NVMe
Policy: Site disaster tolerance = none (single site), Failures to tolerate = 2 failures - RAID-6, IOPS limit none
Tier 2 - Standard (Applications, Mid-priority):
vSAN ESA FTT=1 RAID-5 (4+1 at 6+ hosts), compression enabled
Policy: Failures to tolerate = 1 failure - RAID-5, IOPS limit per VM where noisy neighbours must be contained
Tier 3 - Archive (Backups, Logs):
External NFS or supplemental datastore, array-side data reduction
Policy: array policy, lower priority QoS
SLA Mapping:
Database: 99.99% uptime → Premium tier, FTT=2, host anti-affinity for database nodes, HA restart priority High
Application servers: 99.9% → Standard tier, FTT=1, HA restart priority Medium
Logs and backups: 99% → Archive tier, HA restart priority Low or disabled
Rule: the storage policy is the enforceable form of the SLA — every tier in the requirements has exactly one policy, the policy name carries the tier, and compliance is checked in vSAN health and VCF Operations rather than assumed.
Key Takeaways
- Design Pattern: For stretched cluster across two datacenters, place witness at a third location (cloud, co-location facility, even a small edge appliance). A witness co-located with one of the two data sites cannot arbitrate that site's failure; without a third location, choose active-passive replication instead of a stretched cluster.
Network Design#
NSX Design Fundamentals
NSX abstracts the physical network through virtualization, enabling micro-segmentation and zero-trust security at scale.
NSX Architecture Layers:
Management Plane:
NSX Manager cluster (3-node), handles policy definition, audit logging, REST API
Control Plane:
Central Control Plane (CCP), which runs on the NSX Manager nodes (VCF 9.0 docs p.508) and distributes realised configuration to host and Edge transport nodes; manages segment membership. BGP sessions run on the Tier-0 gateways on the Edge nodes, not in the CCP
Data Plane:
Edge Transport Nodes (Edge VMs or hardware), NSX kernel modules on ESX host transport nodes, forward traffic per policy
Tier-0 & Tier-1 Gateway Design
Tier-0 Gateway:
Top-level logical router, performs North-South routing (external connectivity). Typically placed at network edge.
HA Mode - Active-Standby:
One edge processes traffic, backup takes over on failure. Simple, predictable traffic path, lower CPU on backup node.
HA Mode - Active-Active:
Both edges process traffic simultaneously using ECMP (Equal Cost Multi-Path), requires stateful traffic pinning for flows. Higher throughput (scales linearly with edge count), complex BGP design.
Edge cluster sizing:
Minimum 2 edges (HA pair), typically 2-4 for medium deployments, 4-8 for large. Each edge = dedicated VM (Large: 8 vCPU, 32 GB, 200 GB disk) or bare-metal Edge node.
Tier-1 Gateway:
Logical router below Tier-0, performs East-West routing within segments. One Tier-1 per tenant/workload domain is common. Connects to Tier-0 for external access.
Placement:
Distributed (running on each ESXi host) or centralized (on edge nodes). Distributed scales better, centralized is simpler to troubleshoot.
Failover:
Tier-1 typically advertises routes to Tier-0; if one edge fails, with BFD enabled (default in NSX), failover detection is sub-second (~1.5s with default 500ms interval × 3 multiplier); without BFD, BGP hold timer default is 180 seconds.
Physical Network Design Integration
NSX requires a stable underlying physical network. Design physical topology to support NSX overlay.
Physical Network Topology:
Spine-Leaf (CLOS):
Recommended for modern data centers. Spine switches connect all leaf switches (no leaf-to-leaf links). Any host can reach any other in 2 hops. Supports ECMP load balancing.
3-Tier (Traditional):
Access layer (ToR), aggregation layer, core. More expensive, more oversubscription if not carefully designed.
Underlay Network Requirements for NSX:
MTU:
Minimum 1600 (overlay adds encapsulation headers: Geneve encapsulation adds ~54 bytes (outer Ethernet + outer IP + UDP + Geneve header); 1700 MTU recommended for future header expansion). Set physical NICs to 9000 (jumbo frames) to handle overlays safely.
Bandwidth planning:
East-West overlay traffic adds ~10-15% overhead. Dimension uplinks to carry combined North-South + East-West.
VLAN design:
Separate VLANs for management, vMotion, vSAN, host TEP, Edge TEP and NSX edge uplinks (there are no separate NSX controller VMs; the control plane runs on the NSX Manager nodes). Avoid trunking all VLANs to all hosts (security, VLAN scalability).
QoS:
Tag NSX-managed traffic (DSCP or 802.1p) to ensure control plane (low latency) and data plane (high throughput) are not starved.
NSX Segmentation Design
Segments are logical Layer 2 domains, replacing VLANs. Design segments around security zones and application tiers.
Segment Design Example:
Application Stack Segmentation:
Segment: Production-Web
- Connected to: Tier-1-Production
- Addresses: 10.10.1.0/24
- QoS: High priority (web-facing)
Segment: Production-App
- Connected to: Tier-1-Production
- Addresses: 10.10.2.0/24
- QoS: Medium priority
Segment: Production-Database
- Connected to: Tier-1-Production
- Addresses: 10.10.3.0/24
- QoS: High priority (latency-sensitive)
Multi-Tenant Segmentation:
Tenant-A-Segment (isolated from Tenant-B-Segment via Tier-1 policies)
Uses same IP schema (10.10.0.0/16 internally, isolated at DFW)
Distributed Firewall (DFW) Design
DFW enforces micro-segmentation at the kernel level on each host. Design firewall rules to achieve zero-trust security.
Rule organization:
Rules are processed top-to-bottom; first match wins. Organize by application tier (web → app → database).
Rule type:
Allow-listing (deny-by-default, explicit allows) is more secure than block-listing.
Rule scope:
Can target VMs, segments, security groups, tags, or combinations.
Stateful rules:
DFW maintains connection state; reverse traffic automatically allowed if forward allowed.
Zero-Trust DFW Policy Example:
Rule Set: Production-Zero-Trust
- Rule 1: Allow inbound HTTPS to web servers
- Source: External (any)
- Destination: Segment:Production-Web, VM Tag:web-server
- Service: TCP/443
- Direction: Inbound
- Rule 2: Allow web→app tier
- Source: Segment:Production-Web, VM Tag:web-server
- Destination: Segment:Production-App, VM Tag:app-server
- Service: TCP/8080
- Direction: Inbound (app-server-side)
Rule 3: Allow app→database
- Source: Segment:Production-App, VM Tag:app-server
- Destination: Segment:Production-Database, VM Tag:database
- Service: TCP/5432
- Direction: Inbound
Rule 4: Deny all (explicit — the default DFW rule allows all Layer 2 and Layer 3 traffic, VCF 9.0 docs p.1526, so change the default rule to Drop or add this rule last)
Source: Any
Destination: Any
Service: Any
Action: Drop
Result: Lateral movement is prevented; only east-west flows matching rules are allowed
BGP Design for NSX Edge Routing
NSX edges peer with physical routers via BGP to exchange routes.
BGP neighbor design:
- Peering: each edge node's Tier-0 uplink peers with both top-of-rack switches over two uplink VLANs (two peers per edge). Reserve a local ASN for the edge cluster's Tier-0 and use eBGP to the fabric; the VCF 9.0 design calls for eBGP between tenant and provider gateways to avoid the iBGP full-mesh requirement, so separate ASNs are needed (VCF-WLDNET-LSCA-REQD-CFG).
- Timers: Keep Alive and Hold Down are set per neighbour; Hold Down must be at least three times Keep Alive, and the values must detect peer loss inside the required failover time (VCF 9.0, Configure BGP). Use the short timers the fabric team agrees to rather than the protocol defaults.
- BFD: enable Bidirectional Forwarding Detection on the BGP neighbours for fast link-failure detection (edge VM minimum interval 500 ms, multiplier 3). The VCF 9.0 edge design uses BFD for routing-peering failover in every edge-cluster model except the single stretched cluster that relies on vSphere HA recovery.
- Authentication: MD5 password per neighbour where the fabric mandates it.
- Route policy: advertise only the prefixes the Tier-0 owns (connected Tier-1 segments, NAT and load-balancer VIPs via route redistribution); accept a default route or the enterprise summary from the fabric, never the full table. In active/active Tier-0 designs across two availability zones, use AS-path prepending from the secondary zone when symmetric traffic is required for stateful devices in the path (VCF 9.0 routing option B).
- Graceful restart: Graceful Restart and Helper for peers; not meaningful when a Tier-0 has a single BGP peer (the SR goes down with the peer).
- Document per neighbour: ASN pair, peer IP, VLAN, timers, BFD, password policy and the prefix lists — this table is the design artefact the implementation team deploys from.
Key Takeaways
- DFW Rule Performance: More rules = higher CPU overhead. Consolidate rules using groups and tags. Aim for <100 rules for typical environment. Use rule hit counters to identify unused rules monthly.
Management Domain & Workload Domain Design#
Management Domain Architecture
The management domain is created by the VCF Installer and hosts the VCF Instance's management components: vCenter 9.0, the NSX Manager cluster, SDDC Manager, the fleet management appliance and, for the first instance of a fleet, VCF Operations and VCF Automation (VCF 9.0 Table 2, Types of VCF Domains). It is the control plane of VCF; its availability directly determines whether the infrastructure can be managed.
Management Cluster Sizing:
Minimum size:
4 hosts for the 'high availability' deployment model (three-node NSX Manager, VCF Operations and VCF Automation clusters with anti-affinity, plus one host of restart headroom). A 'simple' deployment is supported from 3 hosts on vSAN (2 on external storage) but leaves no spare host to restore storage-policy compliance after a failure (VCF 9.0 default cluster requirements).
Typical size:
5-7 hosts to allow 1-2 host failures without performance impact on remaining workload domains.
CPU/Memory:
Management cluster does not run user VMs; reserve resources entirely for management services.
vCenter VM sizing:
Size to the inventory: Small 4 vCPU/21 GB (up to 100 hosts or 1,000 VMs), Medium 8 vCPU/30 GB (400 hosts or 4,000 VMs), Large 16 vCPU/39 GB (1,000 hosts or 10,000 VMs), X-Large 24 vCPU/58 GB (2,500 hosts or 45,000 VMs) — vSphere 9.0 docs p.309.
Consolidated vs Standard Architecture:
Standard:
Dedicated management domain; customer workloads run in separate VI workload domains. The recommended production model (VCF 5.2 design guide VCF-ARCH-RCMD-CFG-001: separates management workloads from customer workloads and gives better long-term flexibility and expansion; implication: additional hardware). Also the model for workload provisioning and mobility across VCF Instances.
Consolidated:
Management components and customer workloads share the management domain's default cluster, separated with resource pools. For small-scale environments or an SDDC proof of concept: lower hardware cost, but resource contention and a shared lifecycle for management and workloads. The VCF 9.0 deployment-pattern tables record the trade-off as fast deployment and a smaller footprint against limited scalability compared with the standard architecture.
Identity in the Management Domain
VCF 9.0 ships vCenter 9.0; the external Platform Services Controller was removed in vSphere 7.0 and every vCenter has embedded SSO. Cross-component single sign-on in VCF 9.0 is VCF Single Sign-On through the VCF Identity Broker (embedded as a service on the management-domain vCenter, or deployed as an appliance with a multi-node cluster — VCF 9.0 docs p.517), federating vCenter, NSX, VCF Operations, VCF Automation and the other VCF Operations components to an external identity provider; SDDC Manager and ESX are excluded and keep their own authentication. VCF Single Sign-On is not compatible with vCenter Enhanced Linked Mode.
Multi-vCenter topology:
Each workload domain has its own vCenter deployed into the management domain; they are independent instances, not replicas, and VCF Operations is the fleet-level console across them.
Backup & Disaster Recovery for Management
Management domain failure = entire VCF is unmanageable. Design robust backup strategy.
vCenter backup:
Native backup (HTTPS export), or file-level snapshots to external storage. Test restore procedure quarterly.
NSX Manager backup:
3-node cluster, auto-replicated. Backup one node separately for offsite storage.
SDDC Manager backup:
Includes VCF configuration, cluster info, bundle cache. Critical for day-2 operations.
Backup frequency:
Nightly for vCenter/NSX/SDDC Manager; weekly full backup to offsite storage.
RTO for management:
If management is down, RTO for recovery should be <4 hours. Design backup to support this.
Workload Domain Design
A workload domain is a logical grouping of compute resources (clusters) managed by a single vCenter instance. Design workload domains around organizational or operational boundaries.
VI Workload Domains:
Traditional vSphere clusters for VM workloads.
Single vs multiple domains:
Single domain (all clusters under one vCenter) is simpler. Multiple domains (one vCenter per domain) allows independent scaling, separate security policies, organizational alignment.
When to split into multiple domains:
scale is a late trigger (one X-Large vCenter covers up to 2,500 hosts or 45,000 VMs); the usual triggers are separate business units (cost allocation), different SLAs (upgrade windows), geographic distribution (latency concerns).
Resource boundaries:
Each domain has own storage pools, network segments, security policies. Limited cross-domain sharing.
Kubernetes on VCF (vSphere Supervisor and VKS):
VCF 9.0 has two domain types, the management domain and the workload domain (VCF 9.0 docs p.2978); there is no separate Kubernetes domain type.
Supervisor on a workload domain:
Activate vSphere Supervisor on one or more workload domain clusters; dedicating a workload domain to containers is a design choice, and the same cluster can run VMs and Kubernetes workloads.
Supervisor control plane:
Three control plane VMs in the High Availability Control Plane Model, which can be scaled from the Simple Control Plane Model (VCF 9.0 docs p.592).
VKS clusters (formerly Tanzu Kubernetes clusters):
User-deployed Kubernetes clusters provisioned by the Supervisor in vSphere Namespaces, for microservices.
Multi-AZ (Availability Zone) Design
Multi-AZ architecture deploys workload domains across geographic sites for disaster resilience.
Active-active multi-AZ:
Both sites run production workload, replication is synchronous (RPO=0). Requires <5ms latency, orchestration complexity.
Active-passive multi-AZ:
Primary site active, secondary on standby (via SRM or manual failover). Simpler operationally, higher RPO.
Stretched cluster multi-AZ:
vSAN stretched cluster spans sites; VMs can run on either site transparently. Requires very low latency (<5ms), witness site for quorum.
Key Takeaways
- Management domain sizing: 4 hosts minimum for the high-availability deployment model (3 for a simple deployment on vSAN); the stretched management cluster needs 8 (4 per availability zone). Grow the default cluster for headroom rather than for a VCF host maximum — the cluster limit is the vSphere/vSAN one.
- Workload domain isolation requires separate vSAN clusters per tier (dev/prod/sensitive data) for failure domain segmentation.
- Cross-domain networking: each NSX instance has its own edge cluster and Tier-0 gateway (the management NSX instance is never shared with workload domains); Tier-1 gateways per workload domain or tenant for traffic isolation.
Security Design#
Zero-Trust Architecture with NSX
Zero-trust security assumes no trust by default; every flow must be explicitly authorized. NSX enables zero-trust at the infrastructure layer.
Zero-Trust Design Pillars:
Identity-based access:
VMs authenticated by tags/groups, not IP addresses. Enables workload mobility without firewall reconfiguration.
Least privilege:
Every flow (even east-west) requires explicit allow rule. Deny-by-default posture.
Microsegmentation:
Segment network into small zones; lateral movement is contained.
Continuous monitoring:
DFW flow logs and NSX Intelligence (behavioral analytics) detect anomalies.
DFW Rule Design Methodology
DFW rules are the enforcement mechanism for zero-trust policy. Design rules systematically to avoid gaps.
Rule Design Steps:
Map application architecture (tiers, inter-tier flows).
For each flow, identify source and destination (use VM tags, not IPs).
Determine service (protocol/port) for each flow.
Write explicit allow rule.
Test rule with traceflow before enforcing.
Enable logging on rules for audit compliance.
Review rule hit counters monthly; remove unused rules.
Rule Template:
- Rule Name: Allow-WebServer-to-AppServer
- Source: VM Tag: tier=web
- Destination: VM Tag: tier=app
- Service: TCP/8080
- Action: Allow
- Direction: Both (initiator and responder)
- Logging: On
- Applied To: All Edges and Hosts
NSX Intelligence & Behavioral Analytics
NSX Intelligence (optional module) uses ML to detect anomalous traffic flows and auto-generate recommended firewall rules.
Behavior learning:
Observes flows in "monitor mode" for 2-4 weeks, builds baseline of normal behavior.
Anomaly detection:
Flags flows that deviate from baseline (e.g., web server suddenly making external API calls at 3am).
Rule recommendations:
Auto-suggests DFW rules to allow baseline, or deny anomalies.
Threat intelligence:
Integrates with threat feeds (VMware, third-party) to block known malicious IPs/domains.
Certificate Management Design
NSX components (Manager, Controllers, Edges) require TLS certificates. Design certificate lifecycle to avoid expiry outages.
Certificate authority (CA):
Use internal CA (e.g., Microsoft CA) or public CA. Internal is common for lab/non-public NSX deployments.
Certificate lifecycle:
1-3 year validity. Rotate 30-60 days before expiry. Automate with monitoring.
Multi-node NSX Manager:
All 3 managers need valid certs. If one expires unnoticed, cluster health degrades.
Renewal strategy:
Use certificate monitoring tool (Prometheus, vRealize Operations) to alert 90 days before expiry.
Identity Federation Design
vCenter and NSX can integrate with corporate identity provider (Okta, Azure AD, Ping Identity) for SSO.
vCenter SSO:
Embedded PSC in vCenter 8.x; configure to trust external IdP (OpenLDAP, AD, OIDC).
NSX authentication:
NSX Manager can use local users, LDAP, or OIDC. LDAP is common for AD-integrated environments.
Design consideration:
Keep backup local accounts (vCenter: administrator@vsphere.local, NSX: admin) in case IdP is unavailable.
Role-Based Access Control (RBAC):
Map corporate groups (e.g., "Cloud-Admins", "Database-Ops") to vCenter/NSX roles for delegation.
Compliance Mapping: PCI-DSS, HIPAA, SOC2
Design security controls to satisfy regulatory requirements. Document mapping of controls to design decisions.
PCI-DSS Compliance (Payment Card Industry):
Requirement 1 (Firewall):
DFW rules enforce network segmentation (cardholder data must be isolated).
Requirement 2 (Defaults):
Change default vCenter/NSX passwords, enable RBAC with least privilege.
Requirement 6 (Code security):
VM patching automation, container image scanning (if using K8s).
Requirement 8 (Authentication):
SSO integration, MFA for admin accounts, local account disablement policy.
Requirement 10 (Logging):
vCenter audit logs, DFW flow logs, NSX manager logs sent to SIEM (minimum 1 year retention).
Requirement 11 (Testing):
Quarterly vulnerability scans (ESXi, vCenter), annual penetration test of cardholder network.
HIPAA Compliance (Healthcare):
Encryption at rest:
vSAN encryption, LUKS on Linux VMs, BitLocker on Windows.
Encryption in transit:
vMotion encryption enabled, DPM (Distributed Port Mirror) not allowed (could expose data).
Access controls:
Role-based (clinician vs IT vs admin), audit trail of who accessed what PHI (Protected Health Info).
Logging/audit:
vCenter event database (minimum 1 year), application logs, access logs to all healthcare data.
Breach notification:
Monitoring for data exfiltration (anomalous large data transfers); incident response plan.
SOC2 Type II Compliance (Service Organizations):
Availability:
Demonstrated by SLA uptime, redundancy design (HA, multi-AZ), disaster recovery testing.
Processing integrity:
Configuration management (Terraform, Ansible), change control process (all infrastructure changes logged).
Confidentiality:
Encryption, RBAC, data classification policies.
Security:
DFW micro-segmentation, vulnerability management, threat monitoring.
Privacy:
Data classification and handling rules for personal data, access limited by role, retention and deletion procedures, and evidence that personal data is encrypted at rest (vSAN or VM encryption) and in transit (TLS, encrypted vMotion).
Mapping compliance to VCF controls — design checklist:
- Identity: VCF Single Sign-On to the enterprise identity provider with MFA; role-based access in vCenter, NSX, VCF Operations and VCF Automation; SDDC Manager and ESX accounts rotated by the VCF Operations password manager.
- Segmentation: vDefend Distributed Firewall policy per compliance zone (PCI cardholder segment, PHI segment), default rule Drop after workload onboarding, policy exported as evidence.
- Encryption: vSAN data-at-rest encryption with an external KMIP key provider (or the vSphere Native Key Provider where an external KMS is not mandated), VM encryption for the regulated subset, encrypted vMotion required.
- Audit: vCenter and NSX events, ESX syslog and VCF Operations for Logs forwarded to the SIEM with the retention period the regulation sets.
- Evidence: compliance dashboards in VCF Operations (vSphere and NSX compliance packs) and the design decision register showing which control satisfies which clause.
Key Takeaways
- Compliance Audit Preparation: Maintain a control matrix (Requirement ID → Design Component → Evidence) updated quarterly. Assign ownership (e.g., Security team = DFW rules, Ops team = backup verification).
Availability & Recoverability Design#
RPO/RTO Mapping to Technology
Recovery objectives directly drive architectural choices. Understand the cost/complexity of each recovery strategy.
RPO vs RTO Matrix:
Recovery Objective → Technology Mapping:
Scenario 1: RTO=4hrs, RPO=1hr (Standard SLA 99.9%)
- Technology: Snapshot-based backup, off-site vaults
- Tool: Veeam, Nakivo, or native vCenter snapshots
- Recovery: 3-4 hours to restore, test, validate VMs
- Cost: Moderate backup infrastructure, tape library optional
- Example: Non-critical applications, bulk processing
Scenario 2: RTO=30min, RPO=15min (Advanced SLA 99.95%)
- Technology: vSAN stretched cluster OR VMware Live Site Recovery with replication
- Tool: vSphere Replication (vSAN-to-vSAN RPO as low as 1 minute in VCF 9.0) + Live Site Recovery orchestration
- Recovery: Automated failover <5min, automated networking failover via NSX
- Cost: 2 full datacenters, WAN bandwidth for 15min RPO
- Example: Production applications, customer-facing services
Scenario 3: RTO=minutes, RPO=0 (Mission-critical 99.99%)
- Technology: vSAN synchronous stretched cluster (active-active)
- Tool: vSAN replication (sync), NSX for unified networking
- Recovery: vSphere HA restarts the affected VMs on the surviving site — applications see a restart, not a transparent failover
- Cost: 3 datacenters (two active + witness), low-latency WAN <5ms
- Example: Databases, financial trading, real-time systems
VMware Live Site Recovery (formerly SRM) Design
VMware Live Site Recovery, part of VMware Live Recovery, plans, tests and runs the recovery of VMs between a protected vCenter site and a recovery vCenter site. It coordinates failover of entire VM groups and ensures network reconfiguration.
Protected site:
Primary production environment with VMs and storage replication (vSAN, SRA plugin to external array, vSphere Replication).
Recovery site:
Secondary location with SRM server, placeholder VMs (not running), storage for replicas.
SRM server placement:
One Live Site Recovery instance per site, paired. Recovery plans move VMs from the protected site to the recovery site — as a planned migration, or as disaster recovery when the protected site has failed; afterwards, reprotect reverses protection so the VMs can be failed back.
Recovery plan:
Logical grouping of VMs to failover together (e.g., "App Tier", "Database Tier"). Controls startup order, IP address reconfiguration, network configuration.
Test capability:
SRM can test failover non-disruptively on recovery site (doesn't affect production).
SRM + vSAN Integration:
Storage replication:
vSphere Replication (host-based, asynchronous; vSAN-to-vSAN RPO as low as 1 minute in VCF 9.0, VCF 9.0 docs p.84) or array-based replication through SRAs. Synchronous protection is a vSAN stretched cluster, a separate availability design. Live Site Recovery orchestrates the recovery (plans, retries, cleanup after failover).
Network failover:
SRM customization scripts can trigger NSX segment failover (change next-hop, BGP re-peer), or manually reconfigure routes at recovery site.
Application consistency:
SRM can trigger VSS (Volume Shadow Copy) on Windows VMs before failover to ensure app-level consistency.
Backup Architecture Design
Comprehensive backup strategy combines multiple methods for defense-in-depth.
Backup Tiers:
Tier 1: Continuous snapshots:
vSAN snapshots (on-array), retained for 7-14 days. Fast restore (<1min), protects against accidental deletion/corruption. RPO = seconds.
Tier 2: Daily incremental backups:
Veeam/Nakivo backup appliance, incremental after full backup. Retained 30 days on disk. Restores from full+incrementals take 5-30min depending on size. RPO = 24 hours.
Tier 3: Weekly full offsite backup:
Full backup copied to geographically distant vault (cloud, co-location). Retained 1 year+. RTO for full restore = hours. RPO = 7 days.
Tier 4: Immutable offsite copy:
Monthly full backup to immutable object storage (S3 WORM, Azure Immutable Blobs). Protection against ransomware (cannot be deleted even by admin). RTO = days. RPO = 30 days.
Backup for Management Domain:
vCenter native backup:
HTTPS export snapshot, weekly full, retained 4 weeks locally.
NSX Manager backup:
Cluster backup (all 3 nodes), weekly, offsite copy.
SDDC Manager backup:
Configuration backup, daily, offsite copy.
Target RPO:
If management is lost, recovery from backup should be <2 hours (RTO).
Multi-Site Design Patterns
Different multi-site topologies serve different RTO/RPO targets and operational models.
Active-Passive (Traditional DR):
Primary site runs all production; secondary is dormant (cold standby) or warm standby (replicas kept updated).
RPO: High (depends on replication frequency, typically 1-4 hours for async replication).
RTO: 30min - 4 hours (failover manual or via SRM, network reconfiguration needed).
Cost: Secondary site is underutilized (infrastructure sitting idle).
Use case: Non-critical workloads, cost-sensitive organizations, regulatory requirement for geographic separation.
Active-Active (Load-Distributed):
Both sites run production workload simultaneously. VMs can fail over to other site transparently.
RPO: zero with synchronous replication, which requires < 5 ms RTT between sites (vSAN stretched cluster); beyond that, asynchronous replication with a non-zero RPO.
RTO: <5min (automatic failover via monitoring/orchestration tools).
Cost: Higher (both sites must be sized for full workload), network latency overhead, complex DNS/load balancing.
Use case: High-availability critical apps, elastic scalability, geographic load distribution for latency optimization.
Stretched Cluster (Transparent Failover):
vSAN stretched cluster spans 2 sites with a witness appliance at a third location; one site is the preferred site and both sites are active.
RPO: 0 (every write is committed synchronously to both sites).
RTO: minutes — on a site failure vSphere HA restarts the affected VMs on the surviving site; guest boot and application start add to the figure.
Requirements: inter-site RTT < 5 ms and at least 10 GbE between the data sites; witness reachable from both sites; 8 hosts minimum for a stretched management default cluster (4 per availability zone), 6 for a stretched workload cluster (3 per zone), with the understanding that 3 per zone leaves no automatic rebuild after a local failure (VCF-VSAN-SC-REQD-CFG-006/007).
Cost: two balanced sites plus a witness host; higher than active-passive, lower operational complexity than orchestrated active-active.
Use case: metro or campus distances where zero data loss and automatic site failover are required without application-level clustering.
Choosing between the patterns:
- Start from the business RPO/RTO per workload tier, not from the technology.
- RPO = 0 and RTO measured in minutes with sites under 5 ms apart → stretched cluster.
- RPO of minutes to an hour, RTO of one to four hours, or sites further apart → active-passive with vSphere Replication / VMware Live Site Recovery and recovery plans.
- RPO of hours and a cost ceiling → backup-only cold site.
- Record the rejected alternatives and the latency measurement that ruled them out in the design decision register.
Key Takeaways
- RPO/RTO targets drive replication strategy: synchronous (0 RPO, high latency) vs asynchronous (data loss risk, better latency).
- Backup frequency and retention policy depend on compliance needs — the retention period comes from the applicable regulation or contract (HIPAA: six years for required documentation; ISO/IEC 27001:2022 clause 7.5.3 requires documented information to be controlled for retention and disposition, and Annex A 5.33 requires records to be protected, without fixing a period).
- Site failover decision tree: inter-site RTT < 5 ms (and RPO = 0 required) allows a vSAN stretched cluster with a third-site witness; anything higher points to active-passive asynchronous replication with VMware Live Site Recovery / vSphere Replication.
Design Process Masterclass — From RFP to Implementation#
The VMware Design Framework & Methodology
VCF 9.0 architectures begin with rigorous discovery and structured documentation. The VMware Design Framework (evolved from VVD principles) consists of five phases: Discovery → Requirements → Design → Documentation → Handoff.
Discovery Workshops: The Foundation
Effective discovery captures the true business drivers, not just technology lists.
Requirements Categorization
Organize findings into four requirement classes:
| Category | Examples | Design Impact |
|---|---|---|
| Business | RTO: 4h, RPO: 1h; opex target -30%; platform consolidation on VCF | Drives multi-site vs single-site, backup strategy, automation scope |
| Technical | Performance SLA: <10ms latency for tier-1 app; vSAN stretched cluster; GPU acceleration | Cluster layout, inter-site latency constraints, edge node requirements |
| Compliance | HIPAA: encryption at rest + in transit; audit logging; workload isolation; PCI-DSS: DFW segmentation | Encryption strategy, vDefend rules, namespace design, vTPM requirements |
| Functional | Kubernetes automation; self-service catalog; cost showback; capacity planning integration | VCF Operations dashboard design, API integration, policy framework, telemetry ingestion |
Architecture Artefacts & Their Purpose
- Conceptual Architecture
High-level block diagram showing business domains (e.g., "Development", "Production", "Edge") and their interconnections. No technical depth yet. Audience: CIO, CFO, business stakeholders.
- Logical Architecture
Maps conceptual domains to VCF constructs: which domains become which workload domains (VI workload domains, and which of them run vSphere Supervisor/VKS)? NSX Federation topology (shared Tier-0 vs local T0 per domain)? Security zones (DMZ tier, app tier, database tier, admin tier)? Data flow (east-west, north-south, client-to-service). Audience: architects, security.
- Physical Architecture
Real hardware, IP ranges, VLAN/overlay segment mappings, vSAN disk layouts, NSX node placement, edge location, failure domain boundaries, redundancy paths. Audience: implementation team, NetOps.
Design Decision Register Template
Track every major choice and its rationale:
| Decision ID | Area | Decision | Rationale | Alternatives | Risk Mitigation |
|---|---|---|---|---|---|
| D-001 | Cluster Design | 4-node management + 8-node VI (split) | Isolation for stability; independent scaling | Consolidated | Monitor resource trending |
| D-002 | Storage | vSAN ESA RAID-6 (4+2) FTT=2 (20 hosts) | 67% efficiency with two-failure tolerance; cost per IOPS | RAID-1 FTT=2 (33% efficient) | Keep >= 7 hosts for rebuild headroom |
| D-003 | DR Strategy | Pilot-light site with auto-scale | Cost-optimal 2h RTO; data replication via native | Active-active | Test DR quarterly |
| D-004 | NSX Federation | Shared T0 per VCF instance | Simplified BGP (1 neighbor per site) | Federated T0 | Monitor T0 HA failover |
| D-005 | Encryption | Native KMS on primary, external KMIP 1.1 | Simplicity primary; operational independence DR | All external | Failover runbook testing |
Sizing Tool Outputs → Design Validation
VMware Capacity Planner output feeds into design. Cross-check against:
CPU oversubscription ratio:
Tool says 4:1 for general compute; your design assumes 4:1 → validated.
Memory overhead:
Tool reports 8 GB per host (vmkernel, services); your 512 GB host with 50 VMs gets 10.2 GB avg = 8 overhead, so ~502 GB usable for VMs.
vSAN capacity:
Tool output: "16 TB raw × 0.8 (efficiency) × 0.67 (FTT2 → 2/3 usable) = 8.5 TB per host; 20 hosts = 170 TB usable" — matches your BoM?
Network bandwidth:
Tool flags if East-West traffic (VM-to-VM) exceeds uplink capacity; validate with DFW rules in logical design.
Bill of Materials & Statement of Work Handoff
BoM must be granular and traceable to design decisions:
Management Cluster (4 nodes):
- CPU: 2 × 20-core (40 cores/node) = 160 cores total
- RAM: 384 GB/node (1.5 TB total)
- Storage: 2 × 1.6 TB NVMe (cache) + 4 × 7.68 TB SAS SSD (capacity) — vSAN OSA all-flash, mirrored, FTT1
VI Workload Domain (8 nodes):
- CPU: 2 × 28-core = 56 cores/node × 8 = 448 cores
- RAM: 512 GB/node × 8 = 4.096 TB
- vSAN: 2 × 3.2 TB NVMe (cache) + 8 × 15.36 TB SAS SSD (capacity) per node (all-flash; hybrid HDD capacity is deprecated in VCF 9.0)
NSX (Shared):
- 2 × Edge nodes (Large form factor: 8 vCPU, 32 GB, 200 GB disk each)
- 3 × Segments (2× prod T1, 1× mgmt T1)
Network:
- 2× 25 Gbps for vsan replication
- 4× 10 Gbps for workload traffic (bonded)
Transition to Implementation
Final design doc hands off to Implementation with:
Deployment sequencing:
Rack → cable → BIOS/firmware → ESXi install → vCenter → vSAN formation → NSX bootstrap → workload domain creation → application deployment.
Success criteria:
All health checks green in SDDC Manager and VCF Operations; performance baseline established against the requirements (latency, IOPS, %RDY thresholds from the sizing section); vSAN storage-policy compliance 100%; HA and DRS configured as designed; NSX BGP peers established with BFD up; backup of vCenter, NSX Manager and SDDC Manager completed and a test restore performed; monitoring and alerting live before the first production workload.
Handoff package:
Final design document, design decision register, as-built configuration export, test and validation results, runbooks for day-2 operations, and the open-risk register with owners and review dates.
Key Takeaways
- VCDX Insight: A complete design decision register is non-negotiable. Panelists expect you to explain "why this shape, not that shape?" for every choice. Document assumptions (e.g., "CPU contention acceptable up to 25% RDY") and trade-offs explicitly.
Mathematical Sizing — Show Your Work#
CPU Sizing: Subscription Ratios by Workload Class
Oversubscription is acceptable only when utilization data supports it. The formula:
Total vCPU = (Sum of vCPU requests across all VMs) / Subscription Ratio
Number of Hosts = Total vCPU / (pCPU/host × EVC de-rating × Overhead reservation)
Subscription ratios by workload class (based on utilization patterns):
| Workload Class | Ratio | Justification | Peak CPU %RDY Tolerance |
|---|---|---|---|
| Latency-Sensitive (DB, real-time) | 1:1 | No contention; strict performance SLA | <5% |
| Mixed Tier-1 (ERP, CRM) | 2:1 | Moderate burstiness; tolerate occasional spikes | <15% |
| General Purpose (web, app servers) | 3:1 to 4:1 | Load-balanced; average utilization 20–30% | <25% |
| VDI / Batch | 4:1 to 6:1 | Highly variable; idle periods; DRS consolidation | <30% |
CPU Sizing Example: Mixed Tier-1 Workload
Requirements:
- 120 VMs averaging 6 vCPU each = 720 total vCPU demand
- Workload class: mixed tier-1 (ERP + CRM) → 2:1 subscription
- Hardware: dual-socket, 28-core per socket = 56 pCPU per host
- EVC baseline: Broadwell (legacy requirement) → de-rate to 90% = ~50 effective pCPU
- Overhead reservation: 5% for vmkernel/services = 2.5 pCPU reserved
- Available pCPU for VMs per host: 50 × 0.95 = 47.5 pCPU
Step 1: Apply subscription ratio
vCPU to allocate = 720 vCPU / 2 = 360 pCPU required
Step 2: Number of hosts
Hosts = 360 pCPU / 47.5 pCPU-per-host = 7.58 → round up to 8 hosts
Step 3: Validate headroom
Actual allocation: 8 × 47.5 = 380 pCPU available
Utilization: 360 / 380 = 94.7% → tight, recommend 10 hosts (76% utilization)
Result:
10-host cluster for sustainable 2:1 oversubscription with 24% headroom for growth.
Memory Sizing: The Multi-Layer Math
Memory is the hardest component to right-size because it includes hidden overhead:
Host Memory Budget:
= Σ(VM memory allocations)
+ vmkernel overhead (~8–12 GB per host, varies by ESXi version)
+ vSAN host memory overhead (per disk group / storage pool and device — size it with the vSAN Sizer, VCF 9.0.1 docs p.1679; the OSA cache device is flash, not RAM, p.1653)
+ NSX buffer (if TEP on host: ~2 GB for overlay)
+ HA reservation (either dedicated standby host OR percentage model)
+ 20% headroom for transient spikes
Example for 50 VMs on single host:
- VM memory sum: 50 VMs × 10 GB avg = 500 GB
- vmkernel overhead: 10 GB
- vSAN cache budget: 200 GB raw SSD → 20 GB cache = 20 GB reservation
- NSX buffer: 2 GB
- HA reservation: (500 + 10 + 20 + 2) × 25% = 133 GB (if 25% HA policy)
- Headroom: 20% of above = 133 GB
- Total needed: 500 + 10 + 20 + 2 + 133 + 133 = 798 GB
Physical host size: 1 TB (1,024 GB) ✓
Usable for new VMs: 1024 - 798 = 226 GB headroom
Transparent Page Sharing (TPS) & Memory Compression Impact
Modern ESXi (9.x) uses TPS primarily for security isolation (not performance). Memory compression is more aggressive:
TPS savings:
Typically 5–10% on homogeneous workloads (e.g., 100 identical VDI desktops), negligible on heterogeneous environments. Do NOT rely on TPS for capacity planning.
Memory compression (hypervisor):
ESXi compresses pages before swapping as a last-resort reclamation step (after TPS and ballooning); it is a contention symptom, not a sizing input. Monitor esxtop CMPRS/s and UNCOMP/s and treat non-zero values as an undersized host.
vSAN Capacity Sizing: Raw → Usable Math
Formula (vSAN OSA traditional):
- Usable Capacity = Raw Capacity × Efficiency × Fault Tolerance × Slack Reserve
- where:
- Efficiency = 0.80–0.85 (RAID-1/5/6 overhead)
Fault Tolerance (FTT):
- FTT1 (mirrored): 0.5 (you keep half)
- FTT2 (3 copies): 0.33
- FTT3 (4 copies): 0.25
RAID-6 (striped): ~0.667
NOTE ON SLACK: the flat 25-30% 'slack space' rule applied only BEFORE vSAN 7 U1. From 7 U1 it is Reserved Capacity = Operations Reserve (hardware-dependent, documented examples 6-17%) + Host Rebuild Reserve (~1/N of the cluster: 25% at 4 hosts, ~8% at 12). Combined can be under 10%. With fault domains configured the toggles are unavailable - fall back to ~25% free. VCF 9.1 keeps Reserved Capacity as a design recommendation (VCF-VSAN-RCMD-CFG-003 in the 9.1 design guidance) and adds vSAN ESA Auto RAID 6 and the vSAN Effective Capacity report. Size with the vSAN ReadyNode Sizer, not a flat multiplier. The 0.85 factors below are exam-style arithmetic, not Broadcom guidance.
Slack Reserve = 0.85 (exam-style shorthand; leave 15% unfilled for resync headroom)
- Example: 20-host cluster, 10 TB raw capacity per host, FTT2, RAID-1 mirrored
- Raw: 20 hosts × 10 TB = 200 TB
- Usable = 200 × 0.8 × 0.33 × 0.85 = 45.1 TB
vSAN ESA (9.x):
ESA is single-tier (NVMe only) and uses RAID-5/6 per default, higher efficiency:
Example: 8-host ESA cluster, 8 TB NVMe per host, RAID-6 (4 data + 2 parity; vSAN has no 8+2 layout and RAID-6 needs at least 6 hosts)
Raw: 8 × 8 TB = 64 TB
RAID efficiency: 4/(4+2) = 0.667
Fault tolerance: FTT=2 (two host or device failures)
Slack: 0.85
Usable = 64 × 0.667 × 0.85 ≈ 36.3 TB
Per-host share: 36.3 / 8 ≈ 4.5 TB usable per node
(RAID-5 alternative on the same 8 hosts: ESA uses 4+1 at 6+ hosts, efficiency 0.8, FTT=1 → 64 × 0.8 × 0.85 = 43.5 TB)
vSAN storage clusters (formerly vSAN Max; disaggregated):
Compute and storage clusters are separate. Design for a minimum of four ESX hosts in a vSAN storage cluster, with six recommended (VCF 9.0 docs p.1664, p.1723). The prerequisites for stretching a vSAN storage cluster list three hosts (p.1563); that is a procedure prerequisite, not the design minimum. A storage cluster provides storage to one or more vSAN compute clusters (p.1066), so storage scales independently of compute.
NSX Edge Sizing
Size Edge nodes by form factor and services, not by per-rule formulas:
Form factors:
Edge VM Small (2 vCPU, 4 GB — suitable for lab and proof of concept), Medium (4 vCPU, 8 GB), Large (8 vCPU, 32 GB) and XLarge (16 vCPU, 64 GB), each with a 200 GB disk (VCF 9.0 docs p.1587); bare-metal Edge nodes are the alternative form factor.
DFW does not load the Edge:
Distributed firewall rules are applied in the hypervisor on each VM's virtual interface (VCF 9.0 docs p.2134), so rule count is a host-side design concern, not an Edge throughput input.
What does load the Edge:
North-south routing and stateful gateway services such as gateway firewall, NAT, IDS/IPS and DHCP; service routers scale out by adding Edge nodes to the cluster (p.1869) and check throughput, connection and BGP figures against the NSX configuration maximums (configmax.broadcom.com) for the chosen form factor.
Key Takeaways
- VCDX Insight: Panelists expect you to present the full math, not just the answer. Show your assumptions (subscription ratio, FTT strategy, NVMe model), intermediate steps, and sensitivity analysis (e.g., "if actual utilization is 20% higher, we need X hosts instead of Y"). Defend non-standard choices with the numbers, not with vendor defaults.
Cluster Design Patterns#
Management Cluster: The Foundation
Minimum sizing: 4 nodes (high-availability deployment model)
3-node: supported for a VCF 'simple' deployment on vSAN (FTT=1 RAID-1 works on three hosts), but there is no spare host to restore storage-policy compliance after a failure and no room for the ESA 4+1 layout.
4-node: minimum for the 'high availability' deployment model — tolerates 1 node failure while keeping the three-node NSX/VCF Operations/VCF Automation clusters on separate hosts, with HA admission control space and rebuild capacity.
Larger: 5–7 nodes for large environments (>500 workload VMs) to reduce management overhead impact during failures.
Consolidated vs. Standard Model:
| Model | Configuration | Pros | Cons | Size Threshold |
|---|---|---|---|---|
| Consolidated | 1 cluster: 4 management nodes + customer VMs coexist (resource pools) | Low capex; fast deployment; small footprint | Management node failure impacts workloads; shared lifecycle; limited scalability | Small environments, proof of concept |
| Standard | 4-node management domain + separate VI workload domain (3+ or 4+ nodes) | Isolated; management failure doesn't affect workloads; recommended production model | Higher capex; more hardware | Production, tier-1 SLA, growth expected |
EVC (Enhanced vMotion Compatibility) requirement:
VI Workload Domain: 3 (4 recommended) to 64 hosts for a vSAN cluster (VCF 9.0.1 docs p.454, p.5274); a non-vSAN vSphere cluster can reach 96 hosts (Configuration Maximums, ESX 9.0.0) — see lab vcffts9-03
Sizing guidance:
4–8 hosts:
Single rack, single rack PDU pair. HA admission control via percentage model (25% reserved). DRS fully effective.
8–32 hosts:
Multi-rack; split PDU redundancy across rows. HA admission control: slot-based (requires careful CPU/RAM profile homogeneity) or percentage. DRS with rack affinity rules.
32–64 hosts:
Potential for multiple vSAN racks. Fault domain boundaries via rack/PDU group. DRS with NUMA-aware scheduling.
Stretched VI Cluster (vSAN stretched, inter-site RTT < 5 ms):
Typically 6–12 nodes per site (3 per site minimum, VCF-VSAN-SC-REQD-CFG-007; with 3 per site vSAN cannot rebuild automatically after a local failure).
Witness appliance at a third site, outside both availability zones.
Active-active HA: each site can tolerate 1 node failure independently.
RPO/RTO: 0 RPO (synchronous writes to both sites); RTO = vSphere HA restart plus guest boot after a site failure — minutes, not seconds.
VKS-Dedicated Workload Domain (design choice — still a standard workload domain)
If containerized workloads are a primary use case, isolate in dedicated WLD:
Cluster size:
Size for the Supervisor control plane, the VKS cluster nodes and HA headroom; the Supervisor control plane is three VMs in the High Availability Control Plane Model (VCF 9.0 docs p.592).
vSAN design:
vSAN ESA for performance; smaller disk footprints per VM (Kubernetes workers are lean: 4 vCPU, 8 GB RAM typical).
Network isolation:
Dedicated NSX segments for Kubernetes: kube-control, pod-overlay, ingress, service-lb.
Storage class:
StorageClass → vSAN SPBM + vasa provider ensures Kubernetes PVC maps to correct vSAN datastore.
Namespace isolation:
vDefend micro-segmentation per namespace; DFW rules block cross-namespace traffic by default.
Storage-Dedicated Cluster (vSAN storage cluster, formerly vSAN Max)
Disaggregated architecture: a vSAN storage cluster provides storage resources to one or more vSAN compute clusters (VCF 9.0 docs p.1066):
Cluster composition:
Minimum of four ESX hosts commissioned as vSAN storage hosts, six recommended (p.1664, p.1723); compute hosts separate. The prerequisites for stretching a storage cluster list three hosts (p.1563); treat four as the design minimum. NO VM workloads on storage cluster.
vSAN storage cluster specifics:
Available in VCF 9.0 (not a 9.0.2 addition); erasure-coding policies (RAID-5/6) apply where the host count allows.
Network:
vSAN storage cluster networking: less than 1 ms RTT, minimum 10 GbE, 25 GbE or higher NICs recommended (VCF 9.0 docs p.1616).
Advantage:
Decouple capacity from compute; scale storage independently.
GPU-Enabled Cluster Design
NVIDIA GRID / vGPU for VDI:
Cluster requirement:
All nodes homogeneous GPU model (e.g., all A100-80GB or all L4).
Licensing:
Per-VM vGPU license (tied to vSphere host UUID) + GRID Manager.
Host sizing:
1 GPU per 4–12 VMs (varies by profile: M10 quad-desktop, M60 8-desktop, A100 single high-performance).
vMotion:
Requires EVC CPU profile match + GPU BIOS compatibility; vMotion latency penalty ~500 ms for GPU memory transfer.
HA consideration:
Slot calculation must account for GPU count. A 4-GPU host failure → reserve 4-GPU capacity on another host for HA slots.
AI Enterprise Stack (e.g., inference):
Typically baremetal or container-based (VKS); less common in traditional VMs.
If VM-based: require dedicated cluster to avoid noisy-neighbor contention (GPU-hungry inference can starve other VMs).
Monitor GPU memory with nvidia-smi; set GPU memory reservation in VM hardware profile.
Edge Cluster Sizing
Single Edge Node (lab or small deployment):
A single Edge node is a single point of failure for north-south traffic; use it only where that risk is accepted and recorded.
Form factor: Medium (4 vCPU, 8 GB, 200 GB disk) or larger; Small (2 vCPU, 4 GB) is for lab and proof-of-concept deployments only (VCF 9.0 docs p.1587).
Edge HA Pair (production edge, active-active T0 BGP):
2 Edge nodes, Large (8 vCPU, 32 GB) or XLarge (16 vCPU, 64 GB), each with a 200 GB disk (VCF 9.0 docs p.1587). Check BGP neighbour and connection counts against the NSX configuration maximums for the release (configmax.broadcom.com).
Active-active BGP: Upstream route learning (BGP ECMP) balances ingress traffic; each edge learns full routing table and advertises local T1s.
Failover: sub-second BGP reconvergence; stateful connections may be lost (no NSX Federation backup path).
BGP neighbor calculation:
Edge HA pair, autonomous system: 65000
Upstream routers (core): 4 routers, each in AS 65001
Leaf switches: 8 VLANs, each VLAN has 1 leaf (8 uplinks total)
BGP session count per edge:
- 4 upstream routers (eBGP) = 4 sessions
- 8 leaf switches = 8 sessions
Total: 12 sessions per edge — compare with the BGP neighbour maximum published in the NSX configuration maximums for the release
Multi-AZ vs. Cross-Cluster Stretched: Decision Framework
| Pattern | Topology | Latency RTT | RPO | Capex | Complexity |
|---|---|---|---|---|---|
| Stretched Cluster | 1 vSAN cluster across 2 availability zones, 6 nodes minimum (3 per zone; 8 for the management default cluster) + witness at a third site | <5 ms site to site | 0 (synchronous writes) | Higher (two balanced sites, uniform hardware, witness host) | Medium — one cluster to operate, but storage policies, edge placement and witness connectivity need design care |
| Dedicated cluster per AZ (active-passive DR) | 2 independent clusters, one per site, vSphere Replication / VMware Live Site Recovery between them | Up to 150 ms (replication tolerant) | Minutes (as low as 1 minute for vSAN-to-vSAN replication in VCF 9.0, VCF 9.0 docs p.84) | Medium — DR site sized for protected workloads only | Higher — recovery plans, IP re-mapping, DNS and NSX re-advertisement to test and keep current |
| Separate VCF Instances per site (fleet) | Two complete VCF Instances under one VCF Operations fleet; application-level or replication-based DR | Any | Depends on the replication product per workload | Highest (two full management stacks) | Highest — but full fault isolation between sites and independent lifecycle |
Decision rule: a stretched cluster is the answer only when the business requires RPO = 0 with automatic site failover and the sites are inside the 5 ms budget; otherwise a dedicated cluster per availability zone with orchestrated recovery gives the same site-loss protection at lower cost and with fewer latency-sensitive dependencies. Record the measured RTT, the host-count minimums and the rejected pattern in the decision register.
Key Takeaways
- VCDX Insight: Choose stretched cluster ONLY if latency <5 ms AND synchronized replication RPO is a hard requirement. Multi-AZ is more common for geo-distributed deployments; SRM for cost-optimized DR where RTO/RPO can be 1–4 hours. Panelists will challenge your topology if it doesn't match the business RPO/RTO — know the latency measurement and the host counts behind it.
Availability Design — Aligning to SLAs#
Three 9s, Four 9s, Five 9s: Architecture Mapping
| SLA Tier | Uptime % | Downtime/year | Architecture Minimum | Key Design Points |
|---|---|---|---|---|
| 3-9s | 99.9% | ~8.75 hours | Single site, HA within cluster | 4-node cluster; HA admission control (25% reserved); DRS (avoid consolidation to <3 nodes) |
| 4-9s | 99.99% | ~52 minutes | Stretched cluster OR primary + hot DR | Stretched vSAN + HA, OR separate cluster + NSX Federation + Avi GSLB; zero data loss not required |
| 5-9s | 99.999% | ~5.26 minutes | Multi-site active-active (rare) | Stretched vSAN (synchronous replication) + HA + active-active HA across sites + multi-master NSX Federation; mission-critical only |
HA Admission Control Strategies
Goal: reserve capacity on remaining hosts to guarantee failed-VM restart space.
- Dedicated Host Model
How it works:
Designate 1 host as failover standby; no VMs run on it normally.
Pros:
Simple math; guaranteed capacity; no oversubscription risk.
Cons:
Wastes ~25% capex (1 of 4 hosts idle); poor resource utilization.
When to use:
Small 4-node clusters; mission-critical single-app workloads; cost no concern.
- Percentage-Based Model
How it works:
Reserve X% of cluster CPU/Memory capacity for HA slots (default 25%).
Pros:
Allows hosts to run VMs; dynamic scaling (remove host → reduce reserve).
Cons:
Assumes homogeneous workload; fragmentation if VM sizes vary wildly.
Calculation example:
8-host cluster, 400 GB total memory, 25% reserve = 100 GB reserved, i.e. two hosts' worth of capacity (N+2). The percentage model does not guarantee that the largest VM fits on one surviving host — check that separately (Performance degradation VMs tolerate, and the largest VM against a single host's free memory).
- Slot-Based Model (Advanced)
How it works:
vSphere HA sizes a slot from the largest CPU reservation and the largest memory reservation (plus overhead) among powered-on VMs, not from VM configured size (vSphere 9.0 docs, Slot Size Calculation). Ensure the cluster can accommodate the required slots on N-1 hosts.
Pros:
Most accurate; accounts for actual workload sizes.
Cons:
Recalculation needed if new large VM added; assumes slot sizes don't change.
Example:
Largest reservations = 32 GB memory and CPU equal to 16 cores. Cluster 8 nodes, each 512 GB RAM, 56 cores. Slots per host = min(512/32, 56/16) = min(16, 3.5) = 3 → 7 surviving hosts × 3 = 21 slots. With 80 VMs the cluster cannot pass admission control: one large reservation distorts the slot size, so cap it (das.slotcpuinmhz / das.slotmeminmb) or use the percentage policy.
Pitfall:
Percentage-based HA with DRS "consolidate" aggressive mode can violate admission control. Keep the DRS migration threshold moderate (3 of 5) on HA clusters so DRS does not pack hosts beyond the admission-control reserve.
DRS Rules for HA Resilience
VM-VM Affinity (Should):
Group related VMs (e.g., app tier + cache tier) to minimize vMotion during failure:
Rule: "tier-1-app-vms" affinity "tier-1-cache-vms"
Effect: DRS prefers to run these VMs on same host (shared cache locality reduces latency).
Strength: Should (DRS can violate if capacity constrained).
VM-Host Affinity (Must):
Ensure tier-1 VMs never run on compute-constrained hosts (e.g., dedicated vSAN storage nodes if vSAN Max):
Rule: VM group "prod-vms" must run on host group "compute-hosts"
Effect: DRS rejects vMotion or initial placement on non-compute hosts.
Strength: Must (hard constraint; placement fails if violated).
Anti-Affinity (Should) for HA Resilience:
Spread redundant tier-1 VMs across hosts to tolerate node failure:
Rule: VM "erp-app-01" should NOT run on same host as "erp-app-02", "erp-app-03"
Effect: DRS spreads these VMs; if 1 host fails, 2 copies survive.
Risk: If anti-affinity breached, HA may restart both on same host → potential restart failure if host overloaded.
Mitigation: Combine with percentage HA (25%+ reserved).
Fault Domains: Rack & PDU Separation
vSAN fault domains (single site) group hosts by rack so a rack loss costs at most one copy; a stretched cluster is the two-site special case with the witness at a third location:
Rack-level:
Assign 3 nodes to "rack-A", 3 to "rack-B", 3 to "rack-C" (three fault domains for FTT=1 RAID-1).
PDU-level:
If single rack, ensure nodes split across PDU A + PDU B for power independence.
vSAN implication:
Each fault domain must hold ≥1 copy of data (FTT1). Failure of 1 rack leaves data on other racks.
Fault Domain Configuration (vSphere Client: Cluster > Configure > vSAN > Fault Domains, or the API / PowerCLI):
- Node "esxi-1a" → Fault Domain: RackA
- Node "esxi-1b" → Fault Domain: RackA
- Node "esxi-2a" → Fault Domain: RackB
- Node "esxi-2b" → Fault Domain: RackB
- Node "esxi-3a" → Fault Domain: RackC
Stretched cluster variant: Preferred site = RackA, Secondary site = RackB, witness host at a third location
Proactive HA: Vendor Health-Provider Integration
Proactive HA (introduced in vSphere 6.5 — VMware blog 'vSphere 6.5 new features: HA enhancements'; still documented in vSphere 9.0, Configure Proactive HA) consumes health events from the server vendor's vCenter plug-in (health provider) and acts before a failure:
Feature:
Monitor hard drive predictive failure, power supply degradation, fan failures.
Action:
Trigger proactive VM migration off host BEFORE failure occurs (vMotion to healthy host).
Configuration:
install the server vendor's vSphere Client plug-in (the health provider), enable vSphere DRS, then cluster > Configure > vSphere Availability > Edit > Turn on Proactive HA; choose the automation level (Manual or Automated) and the remediation (Quarantine mode, Mixed, or Maintenance mode) (vSphere 9.0 docs: Configure Proactive HA).
Advantage:
Reduces unplanned HA restarts; maintains performance during proactive migration.
Key Takeaways
- 99.9% availability (SLA) allows ~43 minutes downtime/month: requires 1 N+1 host failure resilience minimum.
- 99.99% availability (four nines) demands N+2 resilience, multi-site failover, and <5 minute detection+recovery time.
- Admission control calculation: Reserve capacity for (Worst-case host failure + peak workload) to maintain SLA during incidents.
Security Architecture Design#
Defense in Depth: Seven Layers
| Layer | Control Category | VCF Technology | Example Rule/Policy |
|---|---|---|---|
| Physical | Access control, intrusion detection | Data center badge, camera, environmental monitoring | Biometric access to server room; 24/7 camera |
| Host/Hypervisor | Secure boot, attestation, access control | ESXi secure boot, TPM 2.0 attestation via HardwareSupport | ESXi lockdown mode (strict); no SSH by default; audit logging all SSH access |
| VM Runtime | VM-level isolation, vTPM, secure boot | vTPM 2.0, vSGX, Virtualization-Based Security (VBS) for Windows | All Windows VMs: require Secure Boot + vTPM + VBS enabled |
| Workload/App | Authentication, authorization, encryption | vDefend microsegmentation, RBAC in app, encrypted credentials | Kubernetes RBAC per namespace; HashiCorp Vault for secret rotation |
| Network | Segmentation, DFW, IDS/IPS | NSX Distributed Firewall, vDefend IDS, AVI WAF | DFW: allow only port 443 (HTTPS) from Tier-1 to Tier-2; IDS: block known CVE-2024 exploits |
| Storage | Encryption at rest, access control | vSAN native encryption, vSAN KMS, SPBM policy | vSAN KMS policy: all tier-1 VMs encrypted; daily key rotation |
| Data in Transit | Encryption, DLP | NSX IPSec, vMotion encryption, vSAN stretched replication TLS | vMotion encrypted (default 9.0); vSAN stretched witness uses mTLS with certificate pinning |
Mapping Controls to Frameworks
NIST Cybersecurity Framework (CSF) Alignment:
Identify:
Asset inventory (CMDB), data classification (PII, confidential, public). VCF: Tags in vCenter, vDefend security groups.
Protect:
Access control (SSO, RBAC), encryption, DFW rules. VCF: vDefend DFW policies tied to security tags.
Detect:
Logging, alerting, IDS. VCF: vDefend IDS inline on all tier-1 traffic; VCF Ops alert on policy violations.
Respond:
Incident response runbook, quarantine VMs. VCF: Automation to isolate compromised VM (move to quarantine VLAN, snapshot).
Recover:
Backup, DR, forensics. VCF: vSAN snapshots, SRM, audit logs → SIEM.
PCI-DSS Zone Design:
DMZ (Untrusted):
Web servers, load balancers exposed to internet. No direct DB access.
App Tier (Trusted):
Application servers, internal APIs. Only HTTP/443 ingress from DMZ.
Database Tier (Secure):
Databases, vault. Only app-specific ports (5432/PostgreSQL, 3306/MySQL, etc.) from App tier.
Admin Tier (Isolated):
Management VMs, backups. Only specific admin users from admin network.
DFW Rule Set for 3-Tier App:
Security Group "sg-dmz" (web servers):
Tag: tier=web, env=prod
Security Group "sg-app" (app servers):
Tag: tier=app, env=prod
Security Group "sg-db" (databases):
Tag: tier=db, env=prod
DFW Rules:
- Allow: sg-dmz → sg-app, Port 443, Protocol TCP (HTTPS)
- Allow: sg-app → sg-db, Port 5432, Protocol TCP (PostgreSQL)
- Deny: sg-dmz → sg-db (zero-trust: no direct web-to-DB)
- (No rule for DB responses — the DFW is stateful and permits return traffic of allowed flows; a DB→app rule would open a new DB-initiated flow)
- Deny: Any other traffic (implicit deny)
vDefend Micro-Segmentation for Zero-Trust
VCF 9.0 vDefend suite includes (previously NSX Security):
DFW (Distributed Firewall):
Inline at hypervisor layer; apply per-vNIC; context-aware rules (user, application, tag).
IDS/IPS (Intrusion Detection/Prevention):
Signature + behavior-based; distributed in the hypervisor at each workload vNIC (Distributed IDS/IPS) or on NSX Edges (Gateway IDS/IPS); inspect encrypted traffic if keys available.
Network Segmentation:
Logical overlay segments (Geneve-backed, VCF-NET-REQD-MTU-006) per tenant/app; microsegmentation via tags.
Zero-Trust Segmentation Model:
Default: DENY ALL traffic
Whitelist: Only explicitly allowed flows (source tag → dest tag, port, protocol)
Example for Kubernetes namespace "production":
Pod Security Group: tag=kube-namespace:production
Egress: Allow to:
- Internal kube-dns (port 53)
- External HTTPS endpoints (port 443, allowlist IPs)
- RDS database (port 5432)
Ingress: Allow from:
- Ingress controller (port 8080)
Result: Compromised pod cannot exfiltrate data laterally to other namespaces
Encryption Design: Key Hierarchy
Native vSAN KMS (Simplest, Recommended for Primary):
Master Key Storage:
TPM-based on vSAN host, or encrypted in vCenter.
Scope:
Per vSAN datastore. All VMs on datastore use same master key.
Advantage:
Operational simplicity; no external KMS dependency.
Disadvantage:
Key tied to vSAN cluster; harder for multi-site key rotation.
External KMIP 1.1 KMS (Recommended for DR/compliance):
Key Management:
Centralized KMIP server (Thales, Gemalto, AWS KMS, Azure Key Vault).
Scope:
Can scope keys per VM or per datastore; multi-cluster support.
Advantage:
Centralized audit trail; key rotation policy enforcement; compliance auditability.
Disadvantage:
Network latency risk (KMS unreachable → cluster degradation); additional operational overhead.
Key Hierarchy Design (Hybrid):
Primary Site:
- KMS Server: Thales KMIP (on-prem)
- vSAN Encryption: Master key cached on hosts, key rotation daily
- VM-Level: Optional AES-256 per VM for HIPAA workloads
DR Site (Pilot-Light):
KMS Server: second KMIP cluster node at the DR site, replicating with the primary (both sites' vCenters trust the same key provider)
vSAN Encryption: the DR cluster has its own cluster KEK from the shared key provider; what matters is that the KEK IDs are served at both sites so hosts can mount disks after a reboot or failover
VM-Level: replicated encrypted VMs keep their keys; the recovery vCenter must reach a key provider holding them before power-on
How vSAN encryption keys work (VCF 9.0, vSAN encryption):
- vCenter requests an AES-256 Key Encryption Key (KEK) from the key provider and stores only its ID.
- Each ESX host encrypts its Data Encryption Keys (DEKs) with the KEK and stores the encrypted DEKs on disk; the KEK itself is never written to disk.
- After a reboot the host asks the key provider for the KEK by ID and does not mount its storage until it arrives — so key-provider availability is a cluster-availability dependency, which is why two key-provider nodes in separate failure domains are a design requirement.
- Shallow rekey issues a new KEK and re-wraps the DEKs (fast); deep rekey also re-encrypts the data (long, I/O heavy) — schedule it against compliance rotation requirements.
- Alternative: the vCenter Native Key Provider needs no external KMS, but ties key custody to vCenter backups; use an external KMIP provider when the regulation requires separation of key custody from the virtualisation platform.
Design decision record (example):
Decision: external KMIP key provider, two nodes across sites, for vSAN data-at-rest encryption on all regulated clusters; Native Key Provider only for non-regulated test clusters.
Justification: HIPAA/PCI key-custody separation; recovery of encrypted clusters at the DR site.
Implication: KMS is a tier-0 dependency — monitored and backed up like vCenter; host reboots stall if it is unreachable.
Alternative rejected: per-VM encryption only (VM Encryption does cover VM files including swap, but leaves non-encrypted VMs and vSAN-level data unprotected and adds per-VM key and storage-policy management).
Key Takeaways
- DFW rule consolidation: use groups and tags to reduce rule count to <100 rules; consolidation improves CPU performance.
- Zero-trust segmentation: design VCF with default-deny policies, explicitly allow required flows (East-West traffic).
- Encryption policy: data-at-rest (vSAN TMK) and data-in-transit (IPsec between sites) required for PCI-DSS compliance.
Multi-Site Design Patterns#
Active-Active (Stretched Cluster)
Topology:
Single vSAN cluster spanning 2 sites; 6 hosts minimum (3 per site) plus a witness appliance at a third site outside both availability zones.
Single HA domain; inter-site RTT < 5 ms (the vSAN stretched-cluster requirement — vMotion itself tolerates up to 150 ms RTT, so it is never the binding constraint).
NSX Tier-0 shared; Tier-1s local or stretched per site.
Characteristics:
RPO:
0 (synchronous replication).
RTO:
minutes — vSphere HA restarts the VMs on the surviving site, then guest OS and application start-up; not sub-minute.
Data locality:
Both sites hold copies; read from local, write synchronously across.
Latency requirement:
<5 ms RTT; lower = better (reduced replication latency).
vSAN Stretched Specifics:
Cluster Configuration:
Fault Domain A (Site-A): esxi-1a, esxi-1b, esxi-1c (3 nodes)
Fault Domain B (Site-B): esxi-2a, esxi-2b, esxi-2c (3 nodes)
Fault Domain C (Witness): witness-node (1 witness appliance; smallest sizes: OSA Tiny 2 vCPU/8 GB, ESA M 4 vCPU/16 GB — VCF 9.0 docs p.1699-1700)
vSAN Policy (stretched):
- Site disaster tolerance: Site mirroring - stretched cluster (one copy per site, witness component in FD C)
- Failures to tolerate: 1 failure - RAID-1 inside each site (FTT defines the additional host failures an object tolerates, VCF 9.0 docs p.1694; with 3 hosts per site a local FTT of 2 is not possible)
- Result: survives the loss of one site plus one host in the surviving site
Witness role:
- Does NOT hold data
- Breaks split-brain (partition → FD with witness wins, other FD fenced)
- If witness fails → cluster continues but loses partition tolerance
Recovery procedure (1 node down):
- Automatic resync; rebuild rate limits to 500 MB/s (adjustable)
- Completion time: (10 TB / 500 MB/s) ÷ 3 (parallel) = ~6-7 hours
- During resync: cluster remains operational, may be slightly slower
Active-Passive (SRM-Based)
Topology:
Production site: active cluster (all VMs running).
DR site: passive cluster (smaller or standby), VMs powered off.
Replication: vSphere Replication (host-based VM replication, VCF 9.0.1 docs p.90) or native vSAN stretched (if <5 ms possible).
Characteristics:
RPO:
15 min – 4 h (depends on snapshot frequency).
RTO:
15 min – 1 h (manual failover via SRM; ~10–30 min to power on VMs and validate).
Cost:
Lower capex (DR site can be smaller; only prod cluster is full-scale).
Complexity:
SRM recovery plan testing required quarterly.
SRM Workflow:
- Create Recovery Plan (in SRM GUI):
- Select VMs to protect
- Define failover priority (tier-1 apps first, batch jobs last)
- Configure failover IP mapping (prod subnet → DR subnet)
- Enable Protection:
- vSphere Replication or array-based replication begins
- RPO monitoring: alert if replication lag exceeds threshold
- Test Failover (monthly):
- SRM creates isolated snapshot copies on DR
- Power on test VMs; validate application functionality
- Cleanup (no impact to production)
- Disaster Event:
- Initiate failover (SRM)
- SRM: powers off prod VMs, powers on DR VMs, updates DNS/load balancer
- Time: ~10–30 min depending on VM count and startup scripts
- Failback (recovery phase):
- Once prod restored, resync replication direction
- Power off DR VMs, power on prod VMs (controlled sequence)
Pilot-Light (Cost-Optimized)
Topology:
Minimal DR site (2–4 nodes); runs non-critical workloads or test VMs.
In disaster, DR site scales up (add cloud resources or pre-staged hardware) and receives production failover.
Characteristics:
RPO:
1–4 h.
RTO:
2–4 h (minimal if pre-staged hardware; longer if cloud scale-up).
Cost:
Lowest capex; minimal licensing on DR site.
Trade-off:
RTO/RPO not as tight as active-active or active-passive.
Automation Example (HCX RAV + Auto-Scale):
Normal state: Prod site (8 nodes), DR site (2 nodes)
Disaster trigger (prod site down):
- Auto-scale: AWS or Azure provisions 6 VMs (cloud-based compute)
- Recovery: VMware Live Recovery / vSphere Replication brings the pilot-light copies to the latest replicated state (HCX RAV is a migration type — Replication Assisted vMotion — not a DR mechanism)
- HCX network extension: stretch DR site subnet to cloud
- Failover: VMs powered on in cloud; restore to pilot-light + cloud hybrid
Cost during disaster: Higher (cloud resources active)
Timeline: RTO 2–4 h (cloud resource provisioning) vs. 30 min if physical standby
Backup-Only (Cost-Minimum)
Topology:
Production site: single cluster or multi-site within site.
DR: nightly backup to off-site storage (tape, cloud object store, or dedupe appliance).
Recovery: restore from backup (no standby cluster).
Characteristics:
RPO:
24 h (1 daily backup).
RTO:
4–8 h (restore + power on).
Cost:
Lowest total cost of ownership.
Risk:
Highest; suitable only for non-critical workloads or dev/test.
Hot / Warm / Cold Tier Decision Matrix
| Tier | Definition | RTO / RPO | Tech | Capex | Use Case |
|---|---|---|---|---|---|
| Hot | DR site fully mirrored, VMs running or ready to restart | Minutes (HA restart) / 0 | Stretched vSAN + vSphere HA, GSLB for application entry points | Highest | Tier-0 SLA (finance, healthcare) |
| Warm | DR site ready but VMs powered off; replication active | 1 h / 15 min (active-passive) | VMware Live Site Recovery, vSphere Replication | Medium | Tier-1 SLA (ERP, CRM) |
| Cold | DR site minimal; backup-only recovery | 4 h / 24 h (backup-only) | Backup platform with off-site copy, tape or object storage | Lowest | Tier-2, dev/test, archive |
Latency Budget Analysis for Stretched Architectures
Binding constraint: vSAN inter-site RTT < 5 ms (VCF 9.0 vSAN network requirements, site to site). vMotion alone tolerates up to 150 ms RTT (long-distance vMotion), so the budget below is spent against the 5 ms vSAN figure.
Network stack contributions to the inter-site round trip:
- Fibre propagation: ~5 µs per km one way (~10 µs per km round trip), so 100 km of fibre path costs ~1 ms RTT before any equipment
- Switch and router latency: a few µs per hop per direction, 3–5 hops between sites — tens of µs in total
- DWDM / transport equipment and encryption appliances: tens to hundreds of µs each
- Host and NIC stack: tens of µs per direction
- Queueing under load: the variable part — measure at peak, not idle
Worked budget: 5 ms vSAN RTT limit − 1 ms safety margin − ~0.3 ms equipment and host overhead = ~3.7 ms for propagation ≈ 370 km of fibre path round trip, i.e. roughly 150–180 km of one-way fibre route (route length, not map distance). Validate with a sustained ping or vmkping test across the actual circuit and look at the standard deviation, not just the mean; jitter above the margin disqualifies the stretched pattern even when the mean passes.
Key Takeaways
- Stretched cluster witness placement: third location (cloud provider, co-location) is mandatory; avoid witness on either primary site.
- Active-passive replication: site affinity rules pin critical VMs to primary site; failover to secondary for DR only.
- Bandwidth planning: vSAN resync uses 10-50% of WAN link; stretched cluster tolerates only 1 component loss, plan link capacity accordingly.
VCF 9.0 Automation & Operations Design Integration#
Catalog Item & Project Hierarchy Design (Upfront Planning)
VCF 9.0 self-service portal requires careful design of project structure, resource quotas, and approval workflows.
Example: SaaS Provider Multi-Tenant Structure
| Org Level | Project/Folder | Resource Quota | Catalog Items Exposed |
|---|---|---|---|
| Root Org | N/A | N/A (system level) | N/A |
| Org: CustomerA | Project: Production | vCPU: 100, RAM: 200 GB, Storage: 5 TB | Web Server, DB Server, Load Balancer (tier-1 items) |
| Project: Development | vCPU: 50, RAM: 100 GB, Storage: 2 TB | Test VM, Sandbox (tier-2 items) | |
| Org: CustomerB | Project: Production | vCPU: 80, RAM: 160 GB, Storage: 4 TB | Web Server, App Server (CustomerB-specific) |
Governance Policies & Customer Org Alignment
RBAC Design (aligned to customer org structure):
AppOwner role:
Can provision VMs from catalog, manage VM lifecycle, view utilization; cannot modify network/storage policies.
Security team role:
Can create DFW rules, vDefend security tags, audit policies; cannot provision VMs.
Ops team role:
Can modify catalog, adjust quotas, manage templates; full platform access.
Policy Enforcement Example (Kubernetes namespace isolation):
Policy: "All tier-1 VMs must have vTPM + Secure Boot + vSAN encryption"
Enforcement: Catalog item "Tier-1 Production VM" template includes:
- vTPM 2.0 device
- UEFI + Secure Boot
- SPBM policy "tier-1-encrypted" (vSAN KMS)
Consequence: User cannot override; if they try manual edit, policy blocks save.
Policy: "Development VMs max 4 vCPU, 16 GB RAM"
Enforcement: Catalog item "Dev VM" defaults capped; quota prevents oversizing.
VCF Ops Dashboard Design for Multiple Personas
Executive Dashboard (C-suite view):
Overall cluster utilization (CPU, Memory, Storage %).
Cost breakdown (capex amortized, opex power/cooling, licenses).
SLA compliance (% uptime, HA events count, planned maintenance windows).
Alerts: only critical (SLA breach, DR site failure, certification expiry).
NOC/Operations Dashboard:
Per-cluster health (vSAN, vCenter, NSX, Edge).
Workload distribution (VMs per cluster, top resource consumers).
Alerts: warnings + critical (DRS recommendations, space trending, failed backups).
Performance metrics (CPU %RDY, Memory contention, IOPS/throughput).
Quick actions: DRS recommendation approval, snapshot trigger, scaling buttons.
Security Dashboard:
DFW rule violations (attempted flows blocked).
vDefend IDS signature hits (top exploits detected).
Compliance status (PCI-DSS controls pass/fail, CIS benchmark scoring).
Encryption audit (VMs without encryption, KMS key age).
Alert-to-Action Chain Design
Automation reduces MTTR (Mean Time To Recovery). Example:
Alert: "vSAN object missing component (FTT1 violated)"
→ VCF Ops detects; triggers automation:
- Increase vSAN rebuild priority (adjust rate limit to 1000 MB/s)
- Migrate non-critical VMs off degraded host (if resync conflicts with workload)
- Notify NOC: "Component missing, auto-rebuild started, ETA 4 hours"
- Update ServiceNow ticket (ITSM integration via webhook)
- If rebuild fails after 2h: escalate to L2 engineer, create incident
Benefit: MTTR reduced from 8h (manual) to 1h (auto-remediation + escalation)
ITSM Integration (ServiceNow) via Webhook
VCF Ops → ServiceNow workflow:
Incident creation:
Critical vSAN alert → auto-create SNOW incident with:
Alert timestamp, severity, component (cluster/host/datastore).
Auto-assign to on-call group (e.g., "vSAN_Team").
SLA: P1 → 1 hour response.
Change approval:
"Cluster upgrade" request in VCF → SNOW change ticket; requires CAB approval before execution.
Feedback loop:
SNOW ticket resolution → VCF Ops cleared incident; closes feedback loop.
Metric Ingestion Design for Custom Metrics
VCF Ops collects native metrics (CPU, memory, storage) but custom app-level metrics enhance visibility.
Integration architecture:
Custom Metric Sources:
- App container (Prometheus) → VCF Ops via vLCM metric adapter
- Database (PostgreSQL pg_stat_statements) → Custom plug-in → VCF Ops
- Load Balancer (Avi) → REST API → VCF Ops
- Example: Track "transaction latency" per application
- Prometheus exports: http_request_duration_seconds (histogram)
- VCF Ops ingests: latency_p95, latency_p99 per app-tier
- Dashboard: shows app latency trend alongside cluster CPU/memory
- Alert rule: "If latency_p95 > 100ms AND cluster_cpu > 80% → recommend scaling"
Key Takeaways
- VCF Operations (formerly vROps) is now mandatory for policy compliance: design must include monitoring and alerting for every SLA metric.
- Ansible integration: design assumes day-2 operations via IaC; document automation dependencies (credentials, API endpoints) in architecture.
- Skyline Advisor: proactive risk identification; plan quarterly reviews to address recommendations before they become incidents.
Documentation Standards — VCDX-Grade Design Docs#
Complete Design Document Outline
A VCDX-grade design document follows this structure:
Executive Summary (1–2 pages)
High-level problem statement, proposed solution, key benefits, cost estimate, timeline.
Audience: CIO, CFO, business stakeholders (non-technical).
Scope (1 page)
What is IN scope: VMs, clusters, NSX, storage, security, HA/DR.
What is OUT of scope: existing legacy systems not migrated, application tuning, networking outside data center.
Example: "This design covers the new production VCF cluster and Tier-1 migration; legacy ESXi 5.5 systems remain in place until 2025."
Assumptions & Constraints (1 page)
Assumptions:
Network latency <5 ms between sites, existing SAN decommissioned by Q3 2025, 2 annual maintenance windows, customer IT staff trained.
Constraints:
Budget cap $2M, must use existing fiber (no new runs), certifications (PCI-DSS tier-1, HIPAA dev tier-2).
Current State Assessment (2–3 pages)
Existing infrastructure: 4 clusters (vSAN 6.x, vCenter 6.7, NSX 3.x).
Utilization snapshot: Avg CPU 35%, Memory 45%, Storage 60%.
Identified pain points: 3 HA failures/month, 2 missed RTO windows/quarter, manual capacity planning.
Conceptual Architecture (2 pages, 1 diagram)
Block diagram: Business domains (Production, Development, DR) → VCF constructs (clusters, NSX zones).
Identify which business requirements map to which domains.
Logical Architecture (5–8 pages, 3+ diagrams)
Cluster layout (Management 4-node, VI 12-node, compute-dedicated, storage-dedicated if Max).
NSX topology (Global Manager, Tier-0, Tier-1s per domain, security zones).
Storage design (vSAN FTT/stripe, vSAN Max disaggregated, datastore capacity).
HA/DR strategy (stretched cluster, SRM, pilot-light detail per domain).
Security architecture (DFW zones, vDefend rules, encryption, compliance mapping).
Physical Architecture (5–8 pages, 3+ diagrams)
Rack layout: ESXi node placement, PDU groups, switch ports, VLAN/overlay segment mappings.
IP addressing scheme: Management network, vMotion, vSAN, NSX overlay, guest VLANs.
Cable plant: uplink bonding (4× 25 Gbps), redundancy (dual switches).
DR site physical layout (if applicable).
Sizing Calculations (5–8 pages)
CPU sizing math (workload class → subscription ratio → host count).
Memory sizing with overhead breakdown.
vSAN capacity (raw → usable), IOPS budget.
NSX edge throughput, edge node count.
HA admission control (slots, percentage reserved, impact).
Growth headroom (next 3-year forecast).
Transition / Migration Plan (3–5 pages)
Phased approach: Phase 1 (Management cluster), Phase 2 (VI cluster), Phase 3 (NSX), Phase 4 (App migration), Phase 5 (Cutover).
Downtime windows (if any), rollback triggers, success criteria per phase.
Parallel run period (if applicable): legacy + new running concurrently for validation.
Operational Plan (3–5 pages)
Runbooks: power-on sequence, failover procedure, recovery from node failure, upgrade process.
Monitoring: alerts, thresholds (e.g., CPU RDY >20% → warning), escalation policy.
Backup/recovery: snapshot frequency, retention, DR test schedule (monthly SRM test).
Capacity management: monthly reporting, quarterly forecasting, expansion triggers.
Exit Criteria / Success Metrics (1 page)
All HA/DR tests pass, SLA compliance >99.9%, performance baselines met (CPU RDY <10%, latency <10ms).
All staff trained, documentation updated, runbooks tested in dry-run.
Appendices
A. Bill of Materials (hardware, licenses, software).
B. Network diagram (detailed).
C. IP addressing spreadsheet.
D. vSAN sizing tool output.
E. DFW rule export (in table format).
F. HA admission control calculation sheet.
G. Design decision register.
H. Risk register with mitigations.
I. Glossary and acronyms.
Section Word Count Guidance
| Section | Target Word Count | Notes |
|---|---|---|
| Executive Summary | 500–1000 | Concise; avoid jargon; focus on business impact. |
| Scope & Assumptions | 800–1200 | Be specific; use bullet lists. |
| Current State | 1000–1500 | Data-driven; include metrics, screenshots, charts. |
| Conceptual Architecture | 800–1200 | 1 diagram; high-level business domains. |
| Logical Architecture | 3000–4000 | 3–4 detailed diagrams (clusters, NSX, security). |
| Physical Architecture | 3000–4000 | Rack diagrams, IP tables, cable specifications. |
| Sizing Calculations | 2000–3000 | Show all math; include examples with numbers. |
| Transition Plan | 2000–2500 | Phased timeline; rollback procedures. |
| Operational Plan | 2000–2500 | Runbooks, alerts, capacity planning. |
| TOTAL | 16,000–20,000 | Professional design doc length. |
Design Decision Register Template
Track every major architecture decision:
| Decision ID | Date | Area | Decision | Rationale | Alternatives | Trade-off | Owner |
|---|---|---|---|---|---|---|---|
| D-001 | 2024-01 | Cluster Design | 4-node management domain + 12-node VI workload domain (standard architecture) | Isolates management from customer workloads; independent scaling and lifecycle | Consolidated architecture | Four extra hosts of capex | Lead architect |
| D-002 | 2024-01 | Storage | vSAN ESA RAID-5 (4+1) FTT=1 on the 12-node VI cluster | Meets the 80% efficiency requirement; ESA builds 4+1 at 6+ hosts | RAID-1 FTT=1 (50% efficient); RAID-6 FTT=2 (67%) | Single-failure tolerance only; rebuild window on large NVMe devices | Storage architect |
| D-003 | 2024-02 | Availability | Dedicated cluster per availability zone with VMware Live Site Recovery (active-passive) | Measured inter-site RTT 7 ms exceeds the 5 ms vSAN stretched-cluster limit | vSAN stretched cluster | RPO of minutes instead of zero; recovery plans to maintain and test | Lead architect |
| D-004 | 2024-02 | Network | Dedicated NSX Manager cluster and edge cluster per workload domain; eBGP with BFD to the fabric | Fault isolation between tenants; fast peering failover | Shared workload NSX Manager | More appliances to patch | Network architect |
| D-005 | 2024-02 | Security | External KMIP key provider (two nodes, one per site) for vSAN encryption | Key custody separate from the platform; DR-site decryption | vCenter Native Key Provider | KMS becomes a tier-0 dependency | Security architect |
Every row must trace back to a requirement ID and forward to the implementation task and the test that proves it. A decision without a documented alternative and trade-off is an assumption, and a VCDX panel will treat it as one.
Document Quality Checklist
- Each requirement has an ID, an owner and a verification method.
- Each design decision cites its requirement(s), states the alternatives considered and the risk accepted.
- Diagrams carry version, date and the cluster/domain names used in the text.
- Sizing shows inputs, formula and result so a reviewer can recompute it.
- Assumptions, constraints and risks are listed in their own registers with review dates.
- Version history records who changed what and why; the current version is the one the implementation team holds.
Key Takeaways
- VCDX Insight: your design documentation must stand on its own without you in the room. Make it scannable: use headings, bullet lists, tables, diagrams. Include a 1-page summary at the front (separate from executive summary) so they can quickly grasp your architecture. If they ask "why did you choose FTT2 for this tier?", the answer must be in the decision register, not improvised.
Lab Exercises & Reference Materials#
Lab Exercise Strategy for VCDX Architect Preparation
Lab exercises bridge theory and real-world architecture. Each lab in this section maps to specific 2V0-13.25 blueprint objectives and uses RCAR (Requirement → Constraint → Assumption → Risk) methodology to build defensible design decisions.
Lab Progression Path:
The labs follow a deliberate progression. Start with the RCAR Design Decision Framework (Lab 01) to establish methodology. Move to Compute Cluster Design (Lab 02) and vSAN Storage Design (Lab 03) for infrastructure foundations. Progress to Multi-Site Stretched Cluster Design (Lab 04) and NSX Micro-Segmentation (Lab 05) for availability and security. Advance to Multi-Workload Domain Design (Lab 06) for enterprise architecture. Then tackle Multi-Site DR Framework (Lab 10) and Self-Service Portal Design (Lab 11) for operational design. Culminate with the Design Document lab (Lab 12) and Healthcare Capstone (Lab 13) that integrate all prior knowledge into VCDX-grade deliverables.
Holodeck Toolkit Lab Environment:
All labs are designed for nested environments using VMware Holodeck Toolkit. Holodeck deploys a complete VCF stack — SDDC Manager, vCenter, ESXi hosts, NSX, vSAN — inside a single physical server. Resource requirements (Holodeck 9.1 docs, single site): VCF 9.0 — 24 (OSA) to 32 (ESA) CPU cores, 325 GB RAM, 1.1 TB SSD/NVMe; VCF 9.1 — 48 cores, 650 GB RAM, 2.2 TB; VCF 5.2 — 16 cores, 384 GB, 2 TB. See the Holodeck section for dual-site figures. For stretched cluster labs, deploy two Holodeck pods with a simulated WAN link using Traffic Control (tc) to introduce 5ms RTT latency. The witness appliance runs as a separate nested VM on a third network segment.
Design Decision Documentation Standard:
Every lab produces design decisions in a consistent format: Decision ID (D-001 through D-NNN), Requirement Addressed (traced to blueprint objective), Options Evaluated (minimum 3 alternatives with pros/cons), Selected Option with justification, Constraints & Assumptions documented, Risk Impact (probability × impact scoring), and VCDX Defense preparation (anticipated panelist questions with pivot responses).
Reference Architecture Documents:
VMware Validated Design (VVD) — historical reference for VCF architecture patterns through VCF 4.x; VMware Cloud Foundation 9.0 Planning & Preparation Guide — current canonical sizing and network requirements; vSAN 9.0 Design & Sizing Guide — ESA storage pool calculations, FTT overhead multipliers, capacity planning worksheets; NSX 9.0 Design Guide — Tier-0/Tier-1 topology, micro-segmentation patterns, BGP/BFD design; VMware Cloud Foundation 9.0 Administration Guide — lifecycle management, upgrade sequencing, SDDC Manager operations.
Exam-Relevant Tools & Utilities:
vSAN Observer — real-time performance monitoring for storage design validation; NSX Traceflow — packet-level verification of DFW rules and routing; SDDC Manager API — programmatic domain deployment and lifecycle operations; Aria Operations — capacity modeling and what-if analysis for sizing decisions; VMware Compatibility Guide — hardware validation for HCL compliance; vSAN ReadyNode Sizer — capacity planning with workload profiles.
Version Compatibility Notes:
The 2V0-13.25 exam targets VCF 9.0 (released 2025). Key version dependencies: vSphere 9.0, vSAN 9.0 (ESA preferred, OSA still a documented storage model), NSX 9.0, SDDC Manager 9.0, VCF Operations 9.0 and VCF Automation 9.0 (the Aria Suite names end with 8.x; Aria Suite Lifecycle has no upgrade path to 9). When referencing older documentation (VVD, VCF 4.x/5.x guides), note the architectural evolution: VCF 5.2 introduced the VCF Import Tool for brownfield adoption and carries the Async Patch Tool for out-of-band critical patches; VCF 9.0 introduced the fleet model (VCF Operations fleet management), made ESA the preferred vSAN architecture and unified the product naming. Labs reference VCF 9.0 constructs but include version context showing evolution from earlier releases.
VCDX Defense Preparation:
Labs 12 and 13 specifically prepare for VCDX design defense. The design document lab (12) produces a complete design with executive summary, sizing calculations, architecture diagrams, 12-decision register, requirement traceability matrix, and risk register. The healthcare capstone (13) simulates a real customer engagement with regulatory constraints (HIPAA/HITRUST), clinical system requirements (Epic EHR, PACS), and multi-site HA/DR — the exact complexity level expected in VCDX submissions. Practice articulating each design decision in under 60 seconds with clear requirement traceability.
Key Takeaways
- Lab progression follows infrastructure-up approach: methodology → compute → storage → network → domain architecture → DR → operations → capstone integration.
- Every lab produces RCAR-formatted design decisions with requirement traceability — the exact format expected in VCDX design document submissions.
- Holodeck Toolkit enables full VCF stack validation in nested environments; stretched cluster labs require two pods with simulated WAN latency.
- Reference materials should be version-pinned to VCF 9.0 exam scope while understanding evolution from VVD/VCF 4.x/5.x for architectural context.
Business Requirements Analysis & Architecture Options (Objectives 1.1, 2.1, 3.1)#
Gathering & Analyzing Business Objectives (Obj 3.1)
Business requirements analysis is the starting point of every VCF design. The architect must translate business language into technical specifications.
Discovery Process:
├── Stakeholder interviews: C-level (budget, timeline, business drivers), IT Director (operational requirements, team skills), Application owners (workload characteristics, SLAs) ├── Current state assessment: inventory existing infrastructure, identify technical debt, document pain points ├── Business drivers documentation: cost reduction targets, compliance mandates, digital transformation goals, cloud strategy ├── SLA decomposition: translate "99.99% uptime" into concrete requirements (RPO, RTO, MTTR, maintenance windows) └── Growth projections: 1-year, 3-year, 5-year capacity planning with seasonal variation
Business vs Technical Requirements (Obj 1.1)
Business Requirements (non-technical stakeholders):
├── "Applications must be available 24x7" → translates to HA design, stretched cluster, SRM ├── "Must comply with data sovereignty regulations" → translates to data residency constraints, specific datacenter locations ├── "Reduce infrastructure costs by 30%" → translates to consolidation ratios, licensing optimization, shared infrastructure ├── "Support 500 new VMs within 6 months" → translates to capacity planning, procurement lead times └── "Enable developer self-service" → translates to VCF Automation, catalog items, approval policies
Technical Requirements (IT stakeholders):
├── CPU/memory/storage sizing for known workloads ├── Network bandwidth, latency, and throughput requirements ├── Integration with existing systems (AD, DNS, NTP, monitoring) ├── Backup and recovery infrastructure compatibility ├── Security posture (encryption, microsegmentation, compliance scanning) └── Automation and orchestration tool integration
Mapping Business to Technical:
Every business requirement must trace to one or more technical design decisions. Use a requirements traceability matrix:
| Business Req ID | Business Requirement | Technical Design Decision | Quality Attribute |
|---|---|---|---|
| BR-001 | 99.99% uptime | vSAN stretched cluster + vSphere HA + SRM | Availability |
| BR-002 | Data in-country only | Single-region deployment, no cross-border replication | Security |
| BR-003 | Developer self-service | VCF Automation with project-level tenancy | Manageability |
VCF Architecture Options (Obj 2.1)
VCF supports multiple deployment architectures depending on scale and requirements:
Standard Architecture:
├── Separate management domain (4 hosts) + one or more workload domains ├── Management domain runs: vCenter, SDDC Manager, NSX Manager, VCF Operations ├── Workload domains run customer VMs ├── Minimum: 4 management + 3 workload = 7 hosts ├── Best for: medium-large environments, multi-tenant, strong isolation └── Trade-off: higher host count, but clean separation of concerns
Consolidated Architecture:
├── Management and workload VMs share the same cluster ├── Single domain with management + workload collocated ├── Minimum: 4 hosts (all running management + workloads) ├── Best for: small environments, edge deployments, lab/dev ├── Trade-off: lower cost, but management contention risk under heavy workload └── Not recommended for production environments with >50 VMs
Multi-Site Architecture:
├── Multiple VCF instances across geographic locations ├── Options: stretched cluster (synchronous), SRM (asynchronous), federation ├── Stretched cluster: RPO=0, RTO<15min, requires <5ms RTT between sites ├── SRM / Live Site Recovery replication: RPO=15min-4hr, RTO=15min-1hr (as in the active-passive pattern), works across WAN ├── Best for: disaster recovery, geographic redundancy, data sovereignty └── Trade-off: complexity + WAN costs vs. availability guarantees
Multi-AZ (Availability Zone) Architecture:
├── VCF 9.0 supports availability zone concept within single management domain ├── Each AZ maps to a fault domain (rack, room, building, site) ├── vSAN fault domains aligned with AZs ├── NSX Edge clusters can span AZs for north-south traffic resilience └── Best for: large single-site deployments with rack-level fault tolerance
Key Takeaways
- Every business requirement must trace to a technical design decision via requirements traceability matrix.
- Discovery process: stakeholder interviews → current state assessment → SLA decomposition → growth projections.
- Standard architecture (separate management/workload) is preferred for production; consolidated only for small/edge deployments.
- Architecture selection driven by: scale, isolation requirements, budget, availability targets, and compliance mandates.
- VCDX expects architects to defend architecture choice with quantified trade-offs, not just preferences.
VCF Consumption Strategy & Automation Tenant Design (Objectives 3.10, 3.11)#
Workload Migration & Onboarding Strategy (Obj 3.10)
Migration strategy defines how workloads move from source to VCF. The architect must plan migration waves, tooling, and validation.
Migration Assessment:
├── Workload inventory: categorize by type (VM, container, physical), criticality (P1-P4), complexity ├── Dependency mapping: application-to-application network flows, shared storage, database dependencies ├── Migration compatibility: HCX-compatible VMs, VMs requiring re-platform, applications needing refactoring ├── Risk categorization: low-risk (stateless web), medium (stateful app), high (database clusters) └── Success criteria per workload: performance baseline, functionality validation, rollback trigger
Migration Approaches:
├── Lift-and-shift (rehost): HCX bulk/RAV migration — fastest, minimal changes ├── Re-platform: move to VCF + modernize (e.g., move DB to vSAN, enable microsegmentation) ├── Refactor: redesign for cloud-native (Kubernetes on VKS/Supervisor) ├── Retire: decommission workloads no longer needed └── Retain: workloads that stay on legacy infrastructure (not VCF-compatible)
Wave Planning:
├── Wave 0: infrastructure (DNS, AD, monitoring) — migrated first or deployed fresh on VCF ├── Wave 1: non-production (dev/test) — validates migration process, trains team ├── Wave 2: low-criticality production — first real workloads with rollback plan ├── Wave 3+: production tiers by application group — migrate dependent apps together └── Final wave: decommission source infrastructure
Onboarding New Workloads:
├── New VMs deployed directly on VCF via vCenter or VCF Automation catalog ├── Container workloads deployed via Supervisor/VKS ├── Network onboarding: NSX segment assignment, DFW rules, load balancer config ├── Storage onboarding: vSAN storage policy assignment (FTT, encryption, compression) └── Monitoring onboarding: VCF Operations adapter auto-discovers new objects
VCF Automation Tenant Design (Obj 3.11)
VCF Automation (formerly vRealize Automation / Aria Automation) enables self-service infrastructure consumption.
9.0 framing: the first tenant-design decision is the organization type. All Apps organizations serve workload domains with Supervisors (VMs and containers): each organization is mapped to one or more vSphere Supervisors through Regions, holds Projects, and each Project holds vSphere Namespaces (VCF 9.0.1 docs p.910). VM Apps organizations serve VM-only clusters with the Aria Automation 8.x-style constructs described below; a workload domain cannot be shared between the two types (p.473). In VM Apps the former Assembler, Service Broker and Orchestrator UIs are merged into one UI — Consume, Design, Content & Policies — while the services remain in the backend (p.112).
Architecture Components:
├── Assembler (formerly Cloud Assembly; Design menu in 9.0, p.112): infrastructure provisioning engine (cloud templates, deployments) ├── Service Broker (Consume and Content & Policies menus in 9.0, p.112): catalog interface for consumers (catalog items, custom forms) ├── VCF Operations orchestrator (embedded, Orchestrator tab, p.112): workflow engine for custom automation ├── Code Stream / Automation Pipelines: not supported in VCF 9.0 (End of Support, release notes p.239) — CI/CD integrates through the API/SDKs and Orchestrator instead └── Salt (SaltStack Config): VCF Automation for VM Apps in 9.0 still offers a SaltStack Config integration (Attach SaltStack Resource day-2 action, available if the integration is configured, VCF 9.0.1 docs p.6793) — confirm licensing and support before designing it in
Multi-Tenant Design (VM Apps organization model; for All Apps use Organization (mapped to Supervisors through Regions, p.909-910) → Project → vSphere Namespace, p.910):
├── Organization: top-level container (All Apps or VM Apps type, p.473)
├── Project: tenant boundary — isolates resources, users, and policies
│ ├── Users/groups: RBAC — administrator, member, viewer roles
│ ├── Cloud zones: resource pools/clusters allocated to the project
│ ├── Network profiles: available network segments for the project
│ ├── Storage profiles: available vSAN policies for the project
│ └── Constraints: tags that control placement (e.g., "zone:prod", "tier:gold")
├── Cloud Zone: maps to vCenter cluster + resource pool + network config
│ ├── Compute: which clusters/resource pools are available
│ ├── Placement policy: default (spread), binpack (consolidate), spread (distribute)
│ └── Capability tags: match deployment constraints (e.g., "gpu:true", "region:us-east")
└── Blueprints/Cloud Templates: YAML-based infrastructure definitions
├── Resources: machines, networks, load balancers, disks
├── Inputs: parameterized values (VM size, OS, network)
├── Constraints: placement tags matching cloud zone capabilities
└── Version control: templates stored in Git, versioned, reviewedSelf-Service & Governance (Obj 3.11):
├── Catalog items: published cloud templates available to project members ├── Custom forms: UI customization for consumer-friendly deployment requests ├── Approval policies: require manager approval for large deployments (>X CPUs, >Y GB) ├── Lease policies: auto-expire deployments after X days (prevents resource sprawl) ├── Day-2 actions: resize, snapshot, power operations available to consumers ├── Quota management: limit CPU/memory/storage per project └── Cost tracking: tag-based chargeback integration with VCF Operations
Automating VCF Infrastructure (Obj 3.11):
├── Cloud templates (YAML): infrastructure-as-code for repeatable deployments ├── ABX actions (Action Based Extensibility): custom code (Python, Node.js) triggered by lifecycle events ├── Orchestrator workflows: complex multi-step automation (e.g., post-deploy AD join, monitoring setup) ├── Event subscriptions: trigger actions on deployment create/update/delete events ├── Terraform provider: manage VCF resources via HashiCorp Terraform └── REST API: full programmatic access to all VCF Automation capabilities
Modern Applications in VCF (Obj 3.11):
├── Supervisor: vSphere-integrated Kubernetes control plane
│ ├── vSphere Namespaces: resource-isolated K8s namespaces backed by vSphere
│ ├── VM Service: deploy VMs as K8s objects (VM Operator)
│ └── Storage: vSAN-backed persistent volumes via CSI driver
├── VKS (vSphere Kubernetes Service): managed Kubernetes clusters
│ ├── Tanzu Kubernetes Grid (TKG) clusters provisioned via Supervisor
│ ├── Lifecycle management: cluster create, scale, upgrade via kubectl
│ └── Registry: Harbor for container image management
├── vSphere Pods: native pod execution on ESXi (CRX micro-VM per pod)
│ ├── No worker VM needed — pods run directly on hypervisor
│ ├── Best for: trusted workloads needing bare-metal-like performance
│ └── Limitation: requires NSX networking, ESA recommended
└── Design considerations:
├── Traditional VMs vs containers: choose based on workload characteristics
├── Stateful workloads: VMs or VMs-as-K8s-objects via VM Service
├── Stateless microservices: VKS clusters or vSphere Pods
└── Hybrid: mix VM and container workloads on same VCF infrastructureKey Takeaways
- Migration wave planning: infrastructure first (Wave 0), dev/test validation (Wave 1), low-criticality production (Wave 2), then critical workloads in dependency-ordered waves.
- VCF Automation tenant model in 9.0: choose the organization type first — All Apps (Organization (mapped to Supervisors through Regions, VCF 9.0.1 docs p.909-910) → Projects → vSphere Namespaces, p.910) or VM Apps (Organization → Project → Cloud Zone → Cloud Templates, with the merged Consume / Design / Content & Policies UI, p.112); a workload domain cannot serve both (p.473).
- Governance pillars: approval policies (gate large deployments), lease policies (prevent sprawl), quotas (resource limits), chargeback (cost visibility).
- Modern app options on VCF: Supervisor (vSphere-native K8s), VKS (managed TKG clusters), vSphere Pods (native pod execution on ESXi), VM Service (VMs as K8s objects).
- VCDX expects consumption strategy to connect business self-service requirements to specific VCF Automation design decisions.
VCF Manageability Design — Objective 3.6 (All Sub-Objectives)#
Manageability determines how efficiently the platform can be operated, maintained, upgraded, and scaled over its lifecycle. The 2V0-13.25 blueprint Objective 3.6 tests manageability across lifecycle management, scalability, and capacity planning.
Obj 3.6.1 — Lifecycle Management Design
SDDC Manager is the lifecycle orchestrator for all VCF components. Design decisions:
Upgrade Strategy: Sequential (management domain first, then workload domains) vs parallel (multiple workload domains simultaneously). Sequential is safer — if management upgrade fails, workload domains are unaffected. Parallel reduces maintenance window duration but increases blast radius.
Bundle Management: Online (SDDC Manager pulls bundles from Broadcom depot) vs offline (bundles downloaded to local repository). Air-gapped environments require offline mode. Decision: maintain a local depot server with pre-validated bundles for controlled rollout.
Pre-check Validation: SDDC Manager runs automated pre-checks before every upgrade — compatibility matrix, disk space, health status, configuration drift. Design: build pre-check into change management workflow. If any pre-check fails, abort and remediate before proceeding.
Upgrade Sequencing for VCF 9.0: Management planes first, then the core SDDC components in a fixed order (VCF 9.0 docs, 'Upgrading a VCF Instance with version 5.x to version 9.0.x'; KB 390634). Management planes: VCF Operations and the VCF Operations fleet management appliance must be on version 9 before the core upgrade; VCF Automation and VCF Operations for networks follow; VCF Operations for logs and VCF Identity Broker are new installations, not upgrades. Core components, per domain: (1) SDDC Manager and VCF services, (2) NSX Manager (and NSX Global Manager, if used), (3) vCenter, (4) ESX hosts (rolling), (5) vSAN on-disk format (if required). Workload domains can be upgraded before or after the management domain, as long as all their components stay compatible. Never skip steps — SDDC Manager validates version compatibility matrix at each stage.
Rollback Strategy: vCenter supports snapshot-based rollback. NSX Manager supports cluster rollback (revert individual nodes). ESXi rolling upgrade allows stopping mid-cluster if issues arise. Design: take vCenter snapshot before upgrade, validate for 24 hours, then remove snapshot. Never run production with active snapshots beyond 72 hours (performance degradation from snapshot delta growth).
Configuration Drift Detection: SDDC Manager tracks desired state vs actual state. If an admin manually changes a setting (e.g., NTP server on ESXi), SDDC Manager flags drift. Design: enforce "no manual changes" policy — all configuration through SDDC Manager API or Automation workflows. Integrate drift detection alerts into VCF Operations dashboard.
Obj 3.6.2 — Scalability Design
Horizontal Scaling: Adding hosts to existing clusters or adding new clusters to workload domains. SDDC Manager automates host commissioning — the architect must design the process: hardware procurement lead time, firmware validation against HCL, network pre-provisioning (ToR switch ports, VLAN trunking), IP address allocation from IPAM.
Vertical Scaling: Not applicable to physical hosts. For VCF management appliances (vCenter, NSX Manager, VCF Operations), scale up by increasing vCPU/RAM allocation during maintenance window. Design: document maximum supported sizes per appliance and trigger points (e.g., "scale vCenter to Large when managed VMs exceed 5000").
Workload Domain Expansion: Adding new workload domains is a major scalability event. Design decisions: new vCenter instance (VCF 5.0+ model), new vSAN cluster, new NSX transport zone (isolated or shared), new IP subnets. Pre-plan 3-5 workload domain expansion slots in network and IP addressing design.
Cluster Scaling Limits: vSAN cluster maximum is 64 hosts. When approaching this limit, design splits into multiple clusters within the same workload domain. Decision: when to split — at 48 hosts (75% capacity) to allow growth headroom without disruption.
Multi-Instance Fleet Scaling: For large enterprises, scale by deploying additional VCF instances managed through Fleet Management. Design: federation topology (hub-spoke vs mesh), shared services (common NSX federation, shared monitoring), operational team assignment per instance.
Obj 3.6.3 — Capacity Planning Design
Capacity Monitoring Integration: VCF Operations provides capacity analytics — time-remaining projections, what-if modeling, and rightsizing recommendations. Design: configure capacity dashboards with 30/60/90-day projection windows. Set alerts at 70% utilization (plan procurement), 80% (expedite procurement), 90% (emergency action).
Growth Modeling: Capacity planning must account for organic growth (existing workloads growing) and project-driven growth (new application deployments). Design: maintain a capacity planning spreadsheet or VCF Operations custom dashboard that combines historical trend data with known project demand from the PMO.
Rightsizing Strategy: VCF Operations identifies oversized VMs (allocated resources significantly exceed actual usage). Design: implement monthly rightsizing review cycle. Create Automation workflow that generates rightsizing report and creates change tickets for approval. Target: reduce wasted capacity by 15-25% annually through rightsizing.
Storage Capacity Triggers: enable vSAN Reserved Capacity (Operations Reserve + Host Rebuild Reserve, VCF-VSAN-RCMD-CFG-003, VCF 9.0.1 docs p.555) — a sliding scale by host count (host rebuild reserve 25% at 4 hosts, 8% at 12); where it is unsupported (3-node, stretched and fault-domain clusters) size slack manually at about 25%. Design: set procurement trigger at 65% vSAN utilization. Emergency trigger at 75%. Never exceed 80% — vSAN performance degrades significantly and rebuild operations may fail.
Capacity Reservation for HA/DR: Capacity planning must account for HA admission control overhead (N+1 = 1/N of each cluster: 25% at 4 hosts, about 6% at 16) and DR site capacity if active-passive. Design: usable capacity = total capacity × (1 - HA_reserve) × (1 - growth_buffer), per cluster. Example: four 16-host clusters (64 hosts) × (1 - 1/16) × 0.80 (20% growth buffer) = 48 hosts worth of workload capacity.
Automation-Driven Capacity Actions: Design closed-loop capacity management: VCF Operations detects threshold → triggers Automation workflow → provisions additional resources (add host, expand cluster) → validates health → notifies operations team. This reduces capacity response time from weeks (manual procurement) to hours (pre-staged hardware) or minutes (elastic cloud burst).
Key Takeaways
- Obj 3.6.1: Lifecycle management via SDDC Manager for the core components and VCF Operations fleet management for the fleet components — upgrade order: management planes, then SDDC Manager → NSX → vCenter → ESX → vSAN on-disk format, offline bundle depot for air-gapped, mandatory pre-check validation, snapshot-based rollback for vCenter.
- Obj 3.6.2: Scalability design — horizontal (add hosts/clusters), workload domain expansion pre-planned in network design, cluster split at 48 hosts (75% of 64 max), multi-instance fleet for enterprise scale.
- Obj 3.6.3: Capacity planning — VCF Operations 30/60/90-day projections, procurement trigger at 70% utilization, vSAN never exceed 80%, rightsizing reviews reduce 15-25% waste, closed-loop automation for capacity actions.
- VCDX Defense: Panelists will ask 'how do you handle Day-2 operations at scale?' — answer with lifecycle automation, capacity triggers, and drift detection. Manageability is the quality that separates a design from a deployment.
VCF Recoverability Design — Objective 3.8 (All Sub-Objectives)#
Recoverability encompasses both business continuity (BC) and disaster recovery (DR). The 2V0-13.25 blueprint Objective 3.8 tests the architect's ability to design recovery strategies that align with business requirements.
Obj 3.8.1 — Business Continuity (BC) Design
Business Continuity vs Disaster Recovery: BC focuses on maintaining operations DURING a disruption (minimizing impact). DR focuses on restoring operations AFTER a disruption (recovery from failure). Both are required — BC reduces RTO exposure, DR provides the recovery mechanism.
BC Requirements Gathering: Interview business stakeholders to determine: Maximum Tolerable Downtime (MTD) per application — how long can the business survive without this application? Recovery Time Objective (RTO) — target time to restore service. Recovery Point Objective (RPO) — maximum acceptable data loss. Minimum Business Continuity Objective (MBCO) — minimum service level during disruption (e.g., "50% capacity is acceptable for 4 hours").
BC Design Patterns for VCF:
Pattern 1 — In-Site Resilience (BC within single datacenter): vSphere HA restarts VMs on surviving hosts after host failure. vSAN FTT ensures data survives disk/host failure. NSX DFW and Edge HA maintain network connectivity. Design: N+1 host capacity, vSAN FTT=1 minimum, dual Edge VMs. RTO: 1-5 minutes (HA restart time). RPO: 0 (no data loss for infrastructure failure).
Pattern 2 — Cross-Site Continuity (BC across datacenters): vSAN stretched cluster provides synchronous replication (RPO=0). vSphere HA spans both sites — VM restarts on surviving site automatically. NSX Tier-0 active-active provides continuous north-south connectivity. Design: identical infrastructure at both sites, < 5ms RTT, witness at third location. RTO: minutes (vSphere HA restart on the surviving site plus guest and application start-up; typically <15 min). RPO: 0. Cost: 2x infrastructure + WAN.
Pattern 3 — Degraded-Mode Operations: Design for graceful degradation when full capacity is unavailable. Example: if Site-A fails, Site-B handles only P1 workloads (50% capacity) while P2-P4 remain offline until Site-A recovers. Design: DRS resource pools with priority levels — P1 VMs get guaranteed resources, P2+ VMs are suspended if capacity is insufficient. Document MBCO per application tier.
Obj 3.8.2 — Disaster Recovery (DR) Design
DR Tier Classification: Align recovery technology to application criticality:
Tier 0 — Zero Data Loss (RPO=0, RTO<15min): Technology: vSAN stretched cluster (synchronous replication). Scope: mission-critical databases, financial systems. Cost: highest (2x infrastructure + low-latency WAN). Design: both sites sized for full workload capacity.
Tier 1 — Near-Zero Loss (RPO<15min, RTO<1hr): Technology: vSphere Replication or vSAN async replication + SRM. Scope: production applications, ERP, CRM. Cost: moderate (DR site can be smaller). Design: SRM recovery plans with automated failover sequencing.
Tier 2 — Standard Recovery (RPO<4hr, RTO<4hr): Technology: backup-based recovery (Veeam, Cohesity) with periodic replication. Scope: supporting applications, reporting systems. Cost: low (backup infrastructure only). Design: daily replication to DR site, SRM or manual failover.
Tier 3 — Best-Effort Recovery (RPO<24hr, RTO<24hr): Technology: nightly backup to offsite storage. Scope: development, test, archive workloads. Cost: minimal. Design: backup-only, no dedicated DR infrastructure.
DR Failover Orchestration: SRM recovery plans define: VM startup priority order (infrastructure first, then databases, then applications, then web tier), IP address remapping (production subnet to DR subnet), custom scripts (DNS update, load balancer reconfiguration, application validation), notification workflow (NOC alert, stakeholder communication).
DR Testing Strategy: Non-disruptive DR test monthly (SRM test failover — creates isolated copies, validates boot/application, cleans up). Full DR test quarterly (actual failover with planned downtime window). Failback test semi-annually (validate reverse replication and failback procedure). Design: document test procedures, success criteria, and results in DR test register.
Obj 3.8.3 — Management Domain Recovery
Management domain loss is the highest-impact failure scenario — all VCF operations depend on it. Design specific recovery procedures:
vCenter Recovery: VCF supports only vSphere HA for vCenter availability. vCenter HA (active/passive/witness) is not supported for a vCenter managed by VCF (VCF 9.0 Table 670, Methods for Protecting vCenter: vSphere HA yes; vCenter HA no; Fault Tolerance no), and even where it exists its recovery time can be up to 15 minutes. Design: vSphere HA on the management cluster with the vCenter VM at high restart priority, plus file-based backup of vCenter to an external location (restore via the vCenter installer; allow 30-60 minutes). Daily automated backup, quarterly restore test.
NSX Manager Recovery: 3-node cluster provides built-in HA. If all 3 nodes lost: restore from backup (configuration + policies). Design: daily NSX backup to external storage, document restore sequence (restore node-1 first, then join node-2 and node-3).
SDDC Manager Recovery: Restore from backup restores VCF configuration, domain inventory, and bundle cache. Design: daily automated backup, separate from vCenter/NSX backup schedule to avoid backup window conflicts.
Full Management Domain DR: If entire management domain is lost (catastrophic site failure): Step 1 — restore SDDC Manager from backup on DR site. Step 2 — restore vCenter from backup. Step 3 — restore NSX Manager cluster. Step 4 — reconnect to surviving workload domains. Step 5 — validate VCF health. Total estimated RTO: 2-4 hours with pre-staged DR hardware. Design: pre-deploy management domain DR hosts, maintain current backups, document and test recovery runbook quarterly.
Key Takeaways
- Obj 3.8.1: Business Continuity — maintain operations during disruption. Key metrics: MTD, RTO, RPO, MBCO. In-site resilience (HA + vSAN FTT), cross-site continuity (stretched cluster), degraded-mode operations (DRS priority pools).
- Obj 3.8.2: DR tier classification — Tier 0 (RPO=0, stretched cluster), Tier 1 (RPO<15min, SRM + async replication), Tier 2 (RPO<4hr, backup replication), Tier 3 (RPO<24hr, backup only). Each tier maps to application criticality and cost.
- Obj 3.8.3: Management domain recovery is highest priority — vSphere HA restart plus file-based backup restore (vCenter HA is not supported in VCF, Table 670), NSX 3-node cluster HA, SDDC Manager backup. Full management DR RTO: 2-4 hours with pre-staged hardware.
- VCDX Defense: Panelists will ask 'what happens when your management domain is lost?' — you must have a documented recovery sequence with realistic RTO estimates. Also expect 'how do you test DR without impacting production?' — answer with SRM non-disruptive test capability.
Modern Applications Architecture on VCF — Objective 3.11 (Design Deep-Dive)#
VCF 9.0 provides a unified platform for both traditional VMs and modern containerized workloads. The 2V0-13.25 blueprint Objective 3.11 tests the architect's ability to design Kubernetes and container infrastructure on VCF.
Supervisor Architecture Design
The Supervisor is the vSphere-integrated Kubernetes control plane. It runs as a set of VMs on ESXi hosts within a workload domain cluster.
Supervisor Deployment Model: Supervisor is enabled per vSphere cluster. Each Supervisor gets 3 control plane VMs (master nodes) distributed across hosts via anti-affinity. Design decisions: which clusters get Supervisor enabled (not all clusters need Kubernetes), sizing of control plane VMs (Tiny: 2 vCPU/8 GB, Small: 4 vCPU/16 GB, Medium: 8 vCPU/16 GB, Large: 16 vCPU/32 GB, each with 32 GB storage; scale-up only, never down — choose based on expected workload count and API request volume).
Networking for Supervisor: Two options — NSX networking (full-featured, recommended) or vSphere Distributed Switch networking (VDS, simpler). NSX networking provides: per-namespace network isolation, NSX load balancer for Kubernetes Services, DFW micro-segmentation for pods, ingress/egress firewall policies. VDS networking provides: basic networking via HAProxy or Avi load balancer, simpler setup but less isolation. Design decision: NSX for production multi-tenant environments, VDS for single-team development clusters.
Storage for Supervisor: vSAN provides persistent storage via the vSphere CSI (Container Storage Interface) driver. Design decisions: StorageClass definitions map to vSAN storage policies. Example: "gold" StorageClass = vSAN FTT=2, RAID-6, encryption enabled; "silver" StorageClass = vSAN FTT=1, RAID-5; "bronze" StorageClass = vSAN FTT=1, RAID-1, no encryption.
vSphere Namespace Design: Namespaces are the tenant boundary on Supervisor. Each namespace gets: resource limits (CPU/memory/storage quotas), network isolation (NSX segment per namespace), storage quotas (per-StorageClass limits), RBAC (mapped to AD/LDAP groups).
Design Pattern — Namespace-per-Team: Create one namespace per development team. Each team gets: resource quota (e.g., 32 vCPU, 128GB RAM, 500GB storage), permission to deploy TKG clusters and VMs within their namespace, isolated network segment, access to approved StorageClasses. This mirrors the VCF Automation project model for VM consumers.
VKS (vSphere Kubernetes Service) Design
VKS replaces Tanzu Kubernetes Grid (TKG) as the managed Kubernetes offering on VCF 9.0.
TKG Cluster Architecture: TKG clusters are deployed within vSphere Namespaces. Each TKG cluster has: control plane nodes (1 or 3, depending on HA requirement), worker nodes (1-N, auto-scalable), all running as VMs on ESXi via the Supervisor.
Cluster Sizing Design: Small development cluster: 1 control plane + 3 workers (8 vCPU, 16GB each). Production cluster: 3 control plane + 6-12 workers (16 vCPU, 32GB each). Design decision: worker node VM class selection — map to workload requirements (CPU-intensive, memory-intensive, GPU-enabled).
Multi-Cluster Strategy: Design decision: single large cluster vs multiple smaller clusters. Single large cluster: simpler operations, namespace isolation within cluster, risk of blast radius. Multiple clusters: environment isolation (dev/staging/prod), workload isolation (batch vs real-time), upgrade independence. Recommendation: separate clusters per environment, shared registry and CI/CD pipeline.
VM Service — VMs as Kubernetes Objects
VM Service allows deploying traditional VMs through the Kubernetes API using VM Operator. This bridges the gap between VM and container operations.
Use Cases: Legacy applications that cannot be containerized but need to be managed alongside containers. Database VMs that provide backend services to containerized frontends. Windows workloads in a predominantly Linux/container environment.
Design: Define VM Classes that map to hardware profiles (CPU/memory combinations). Create ContentLibrary sources for VM images (OVA/OVF templates). Deploy VMs using kubectl with YAML manifests — same GitOps workflow as containers.
Container Registry Design (Harbor)
Harbor provides container image management for VCF Kubernetes environments.
Deployment Model: Embedded Harbor (deployed automatically with Supervisor) for small environments. External Harbor (standalone deployment) for enterprise environments needing HA, multi-site replication, and advanced scanning.
Image Security: Harbor integrates vulnerability scanning (Trivy). Design: enforce "no deploy without scan" policy — Kubernetes admission controller rejects images with critical CVEs. Image signing with Notary/Cosign for supply chain security.
Design Pattern — Registry Hierarchy: Global registry (Harbor) hosts golden images (base OS, middleware). Team registries (project-level) host application images built from golden bases. Production promotion: images promoted from dev → staging → prod registries with approval gate.
Network Design for Modern Apps
Kubernetes networking on VCF requires careful design:
Pod Networking: With NSX, each pod gets an IP from the namespace's NSX segment. Pod-to-pod communication within namespace: direct L2. Cross-namespace: routed through Tier-1 gateway with DFW policy enforcement.
Service Networking: Kubernetes Services (ClusterIP, NodePort, LoadBalancer) mapped to NSX or Avi load balancer. Design: use NSX load balancer for L4 services, Avi for L7 (HTTP path-based routing, WAF, SSL termination).
Ingress Design: NSX or Avi Ingress controller. Design: single ingress controller per cluster, wildcard TLS certificate, path-based routing to backend services. For multi-cluster: use Avi GSLB for cross-cluster load balancing.
Network Policy: Kubernetes NetworkPolicy objects enforced by NSX DFW. Design: default-deny ingress per namespace, explicit allow rules per application. This extends the zero-trust model from VMs to containers.
Key Takeaways
- Obj 3.11: Supervisor = vSphere-integrated K8s control plane. 3 control plane VMs per cluster. NSX networking (production, multi-tenant) vs VDS (simple, single-team). vSphere CSI for persistent storage.
- Namespace-per-team design pattern mirrors VCF Automation project model. Each namespace gets: resource quota, network isolation (NSX segment), storage quota, RBAC mapped to AD/LDAP.
- VKS cluster strategy: separate clusters per environment (dev/staging/prod) for upgrade independence and blast radius containment. VM Service bridges VM and container operations via kubectl.
- Container security: Harbor registry with Trivy scanning, admission controller blocks unscanned images, image signing for supply chain security. Registry hierarchy: global golden images → team registries → production promotion.
- VCDX Defense: Expect 'how do you handle mixed VM and container workloads?' — answer with Supervisor + VM Service + VKS on shared VCF infrastructure, unified monitoring via VCF Operations, NSX providing consistent network policy across both.
VCF Monitoring & Observability Design — Objective 3.12 (All Sub-Objectives)#
Monitoring design ensures visibility into platform health, workload performance, and operational compliance. The 2V0-13.25 blueprint Objective 3.12 tests the architect's ability to design monitoring for both management components and workloads.
Obj 3.12.1 — Management Component Monitoring
VCF management components are the foundation — if monitoring fails to detect management plane issues, all downstream workloads are at risk.
SDDC Manager Monitoring: Monitor SDDC Manager health via API (/v1/system/health). Key metrics: task queue depth (stuck tasks indicate lifecycle issues), bundle download status, certificate expiry warnings, configuration drift count. Design: VCF Operations adapter for SDDC Manager, alerts on task failures and certificate expiry (90-day warning, 30-day critical).
vCenter Monitoring: Key metrics: VPXD service health, SSO authentication latency, database size and performance, managed object count (VMs, hosts, clusters). Critical alerts: VPXD service restart, SSO token failures, database backup failures. Design: VCF Operations vCenter adapter + custom dashboards for admin team. Monitor the vCenter VM's vSphere HA protection state and restart events (vCenter HA is not supported in VCF, Table 670) — alert if the VM is restarted or unprotected.
NSX Manager Monitoring: Key metrics: cluster health (all 3 nodes UP), API response latency, control plane connectivity to transport nodes, certificate validity. Critical alerts: NSX Manager node failure (cluster degrades from 3→2), transport node disconnection (host loses overlay networking), DFW rule publish failures. Design: NSX adapter in VCF Operations, correlated alerts (NSX node down → check host connectivity → check network health).
ESXi Host Monitoring: Key metrics: CPU ready time (%RDY — performance indicator), memory ballooning/swapping (contention indicator), vSAN disk health and latency, network packet loss/errors, hardware health (sensors via IPMI/iLO/iDRAC). Critical alerts: host disconnected, hardware predictive failure (Proactive HA trigger), vSAN disk failure, network link down. Design: per-host health scorecard in VCF Operations, aggregated cluster health view.
vSAN Monitoring: Key metrics: cluster health (vSAN Health Service), resync/rebalance status, object compliance (all objects meeting storage policy), disk latency (read/write), cache hit ratio (OSA), capacity utilization. Critical alerts: object non-compliance (data at risk), resync stalled, capacity > 80%, disk failure. Design: vSAN adapter in VCF Operations + vSAN Health Service native alerts. Include vSAN performance trending for capacity planning.
Obj 3.12.2 — Workload Monitoring
Application-Level Monitoring: VCF Operations monitors infrastructure metrics (CPU, memory, storage, network) for each VM. For deeper application insight, integrate with application performance monitoring (APM): Telegraf agent in VMs exporting custom metrics, VCF Operations custom metric ingestion via REST API, correlation between infrastructure metrics and application KPIs (e.g., high CPU → slow response time).
Kubernetes Workload Monitoring: For Supervisor and VKS clusters: pod/container resource utilization (CPU/memory requests vs limits vs actual), persistent volume capacity and IOPS, Kubernetes API server health, etcd cluster health and latency, node readiness and conditions. Design: VCF Operations Kubernetes adapter + Prometheus integration for custom application metrics. Dashboard: cluster overview → namespace drill-down → pod detail.
Monitoring Topology Design:
Centralized Model: Single VCF Operations instance monitors all VCF instances, workload domains, and applications. Advantages: single pane of glass, unified alerting, simplified operations. Disadvantages: single point of failure for monitoring, potential performance bottleneck for large environments (>10,000 VMs). Design: deploy VCF Operations in HA mode (2-node minimum), place on management domain for maximum reliability.
Distributed Model: VCF Operations instance per VCF instance or per geographic site. Central analytics node aggregates data from distributed collectors. Advantages: monitoring survives site failure, lower WAN bandwidth for metrics, local team autonomy. Disadvantages: more complex operations, potential inconsistency in alert definitions. Design: remote collector nodes at each site, central analytics in primary datacenter.
Hybrid Model (Recommended for Enterprise): Central VCF Operations cluster in primary datacenter. Remote collectors at each remote site/VCF instance. Cloud proxy for Broadcom SaaS analytics (if internet-connected). Design: central cluster sized for total VM count, remote collectors sized for local VM count.
Obj 3.12.3 — Alerting & Escalation Design
Alert Classification: Informational (no action, logging only) → Warning (investigation needed within 4 hours) → Critical (immediate action required, SLA at risk) → Emergency (service down, escalation to management).
Alert Correlation: VCF Operations supports alert correlation — multiple related alerts grouped into a single incident. Design: define correlation rules to reduce alert noise. Example: if "ESXi host disconnected" + "vSAN object non-compliant" + "VM powered off" all fire within 5 minutes → correlate to single incident "Host failure with data impact."
Notification Channels: Email (all severities), Slack/Teams webhook (warning and above), PagerDuty/OpsGenie (critical and emergency), ServiceNow incident creation (critical and emergency). Design: notification routing by team — infrastructure alerts to platform team, application alerts to application owners, security alerts to SOC.
Escalation Matrix Design:
Level 1 (L1) — NOC On-Call: Responds to warnings within 30 minutes. Runs documented remediation runbooks. Escalates to L2 if not resolved within 1 hour.
Level 2 (L2) — Platform Engineer: Responds to critical alerts within 15 minutes. Deep troubleshooting using VCF Operations, esxtop, vSAN Observer, NSX Traceflow. Escalates to L3 if not resolved within 2 hours.
Level 3 (L3) — Architect/Vendor Support: Responds to emergency alerts immediately. Engages Broadcom GSS (Global Support Services) for product defects. Coordinates with application teams for cross-functional issues. Post-incident review and design improvement.
Obj 3.12.4 — Compliance & Audit Monitoring
CIS Benchmark Monitoring: VCF Operations includes CIS compliance dashboards for ESXi 8.x hardening benchmark. Design: enable CIS benchmark content pack, schedule weekly compliance scans, alert on any non-compliant findings. Track compliance score trending (target: >95% compliance).
Audit Log Design: All VCF components generate audit logs — vCenter events, NSX audit log, SDDC Manager task log, ESXi shell access log. Design: centralize all logs to VCF Operations for Logs (or external SIEM — Splunk, QRadar, Elastic). Retention: minimum 1 year for compliance (PCI-DSS, HIPAA), 90 days online searchable, remainder in cold storage.
Change Tracking: Monitor for unauthorized changes — ESXi configuration changes outside SDDC Manager, NSX DFW rule modifications, vCenter permission changes. Design: VCF Operations drift detection alerts + integration with change management system (ServiceNow). Any change without an approved change ticket triggers security alert.
Dashboard Design for Compliance Officers: Executive compliance dashboard showing: overall compliance posture (% compliant), top non-compliant findings with remediation status, audit readiness score, certificate expiry timeline, encryption coverage (% of VMs/datastores encrypted). Design: read-only access for compliance team, scheduled PDF reports (weekly), drill-down capability to specific control failures.
Key Takeaways
- Obj 3.12.1: Management component monitoring — SDDC Manager (task queue, cert expiry), vCenter (VPXD health, HA state), NSX (cluster health, transport node connectivity), ESXi (CPU %RDY, hardware health), vSAN (object compliance, capacity, latency).
- Obj 3.12.2: Workload monitoring — infrastructure metrics via VCF Operations adapters, application metrics via Telegraf/custom ingestion, Kubernetes monitoring via K8s adapter + Prometheus integration.
- Obj 3.12.3: Alerting design — 4-level classification (info/warning/critical/emergency), alert correlation reduces noise, notification routing by team, L1/L2/L3 escalation matrix with defined response times.
- Obj 3.12.4: Compliance monitoring — CIS benchmark scanning, audit log centralization (1-year retention minimum), configuration drift detection, executive compliance dashboards with remediation tracking.
- VCDX Defense: Panelists will ask 'how do you know when something is wrong?' — answer with layered monitoring (management → infrastructure → workload → application), correlated alerting, and closed-loop remediation via Automation integration.
Exam Mapping: 2V0-13.25 — VCP-VCF 9.0 Architect
- See VCP-VCF 9.0 Architect exam blueprint for detailed objectives
Labs in This Section
📝 Quiz — VCF 9.0 Architect
Section 1 — Architecture Design Methodology
- A technology-specific diagram showing vCenter, NSX Manager, and vSAN relationships but no specific vendor hardware
- A spreadsheet of business requirements translated into user stories
- A high-level diagram aligned to business goals showing only capability boxes without technology
- A bill of materials listing specific server SKUs, firmware versions, and switch part numbers
- Assumption
- Constraint
- Requirement
- Risk
- Availability
- Performance
- Recoverability
- Manageability
- Decision, cost, vendor
- Owner, date, and approver only
- Justification, implications, and alternatives considered
- A diagram and a configuration file
- A risk register entry
- A validation checklist failure
- An AMPRS trade-off between Availability and a cost constraint
- A functional vs. non-functional requirement conflict