Academy/NSX 4.x Network Virtualization Professional (2V0-41.24)/Lab N1: Deploy DPDK-Accelerated NSX Edge Cluster in VCF 9.0
This lab targets VCF 9.0

Lab N1: Deploy DPDK-Accelerated NSX Edge Cluster in VCF 9.0

VCF 9.0Advancedvcp-foundation⏱ 120 min

NSX 9.0.x (feature set inherited from the NSX 4.2 line; NSX 4.2 docs remain a valid technical reference) Edge cluster deployment — Edge DPDK fastpath interfaces, host-switch Enhanced Data Path, Edge sizing, N-S throughput validation

Objectives

  • Understand NSX Edge node architecture (DPDK fastpath interfaces) and host-side Enhanced Data Path
  • Deploy and configure an Edge cluster suitable for high-throughput N-S traffic
  • Configure Edge transport nodes with proper CPU reservation and NUMA alignment
  • Validate Edge cluster health and throughput performance
  • Understand Edge sizing tiers and their impact on supported services

Prerequisites

VCF 9.0 management domain operational, dedicated ESXi hosts for Edge VMs (or shared hosts with sufficient resources), VLAN uplink segment configured for Edge external connectivity, overlay transport zone operational

Prior labs: net-virt-01

Required skills:

  • NSX Edge architecture concepts
  • DPDK/SR-IOV fundamentals
  • vSphere resource management (CPU reservation, NUMA)

Lab Environment

VCF 9.0 with 2+ ESXi hosts designated for Edge workloads. Each host should have sufficient resources for Edge VMs: Large form factor = 8 vCPU, 32 GB RAM per Edge VM. Physical NICs connected to upstream switches for Edge uplink VLANs.

Tasks

Task 1 Deploy and Validate DPDK-Accelerated Edge Cluster

Edge sizing in VCF 9.0 follows predefined form factors: Small (2 vCPU/4GB — lab only), Medium (4 vCPU/8GB — small deployments), Large (8 vCPU/32GB — production standard), Extra Large (16+ vCPU/64GB — high throughput). DPDK fastpath bypasses the kernel network stack, dedicating CPU cores to packet processing via poll-mode drivers. In VCF 9.0, Edge clusters are deployed from the vSphere Client (vCenter → Networks → Network Connectivity → Configure Network Connectivity, Centralized Connectivity; VCF 9.0.1 docs p.1576-1577) — this lab walks through the NSX Manager process for understanding.

Deploy NSX Edge nodes with the Enhanced Data Path (DPDK fastpath) for maximum north-south throughput, configure proper resource reservations, form an Edge cluster, and validate performance under load. Understanding Edge architecture is critical for VCDX — Edge sizing directly impacts north-south bandwidth, stateful service capacity, and failure domain design.

Step 1

Plan Edge host resources. NSX Edge VMs require: (a) CPU reservation — 100% for all vCPUs (DPDK poll-mode drivers consume CPU continuously even at idle); (b) Memory reservation — 100% (no overcommit); (c) NUMA alignment — all vCPUs and memory on a single NUMA node for optimal DPDK performance; (d) Latency sensitivity — set to 'High' in VM Advanced settings. For a 4-Edge cluster on 2 hosts (anti-affinity), each host needs: 2 × 8 vCPU = 16 cores + 2 × 32 GB = 64 GB reserved.

Step 2

Prepare the Edge prerequisites. Verify (or create) an Edge uplink profile (NSX Manager → System → Fabric → Profiles → Uplink Profiles) with at least 2 uplinks for redundancy (failover order or load-balance source teaming, depending on physical topology), plus the overlay transport zone, the VLAN transport zone for uplinks and the TEP IP pool. Transport node profiles apply only to ESX hosts in a vCenter cluster and cannot be applied to NSX Edge nodes (VCF 9.0.1 docs p.2007); these Edge settings are entered per Edge node in step 3.

Step 3

Deploy Edge nodes via NSX Manager. System → Fabric → Edge Transport Nodes → Add Edge VM. Configure: Name='edge-node-01', Form Factor=Large, Host=esxi-edge-01 (place on dedicated Edge host), Datastore=select shared or local datastore, Management Network=management port group. Configure uplinks: Edge uplink 1 → VLAN uplink segment (toward physical router), Edge overlay TEP → overlay TEP segment. Repeat for edge-node-02 on esxi-edge-02 (anti-affinity).

Step 4

Verify Edge node deployment. Wait for deployment to complete (5-10 minutes per Edge). In NSX Manager → System → Fabric → Edge Transport Nodes, verify: Configuration State='Success', Node Status='Up', Manager Connectivity='Up', Controller Connectivity='Up'. If any show 'Failed', check: (a) Edge VM console for boot errors, (b) management network connectivity from Edge VM, (c) DNS resolution from Edge (required for Manager/Controller communication).

Step 5

Form Edge cluster. System → Fabric → Edge Clusters → Add Edge Cluster. Name='edge-cluster-prod', Member Type=Edge Node, add edge-node-01 and edge-node-02 (and edge-node-03/04 for production). For DPDK high-availability: set the Edge cluster profile with allocation rules — 'AllocationBasedOnFailureDomain' distributes T0 service instances across failure domains.

Step 6

Verify the DPDK fastpath. SSH to edge-node-01 as admin: get service dataplane (state should be running); get interfaces and locate fp-eth0 (overlay) and fp-eth1/fp-eth2 (external) — the fastpath interfaces are DPDK-backed on every Edge VM form factor, there is no 'standard' Edge datapath to switch on; get dataplane | find ring shows the rx/tx ring sizes (128-4096). Separately, in NSX Manager confirm the host switch of the Edge hosts runs Enhanced Data Path (EDP Standard is enabled automatically in new VCF 9 workload domains).

Step 7

Configure anti-affinity rules. In vSphere Client, create a DRS anti-affinity rule: Name='Edge-Anti-Affinity', Members=edge-node-01, edge-node-02 (and 03/04). This ensures Edge VMs never co-locate on the same host, maintaining N+1 availability. If the Edge cluster was deployed through the vCenter Network Connectivity workflow, check whether a rule already exists before you create one (the VCF 9.0.1 docs do not state that this workflow creates it).

Step 8

Performance baseline. Attach iperf3 server on the external network (beyond the physical router). From a VM on the overlay: iperf3 -c <external-iperf-server> -P 8 -t 30 (8 parallel flows, 30 seconds). Record the baseline and compare it with the sizing guidance: a Large Edge is suggested for a total throughput requirement between 2 and 10 Gbps (VCF 9.0.1 docs p.2043); above that, plan Extra Large or more Edge nodes in Active-Active (ECMP). Monitor the Edge during the test: get service dataplane stays running and the Edge VM's CPU usage in vCenter rises on the dataplane cores (poll-mode drivers already hold them near 100 % at idle, so judge by packet counters on get interfaces, not by CPU %).

Step 9

Edge failure simulation. Gracefully shut down edge-node-01 VM. Monitor: (a) T0 BGP neighbor state — should transition from Established to the backup Edge node within BFD timeout (sub-second with BFD, 12-180s without); (b) iperf3 traffic — brief interruption followed by recovery on edge-node-02; (c) NSX Manager → Networking → Tier-0 → edge-node-01 shows 'Down', edge-node-02 shows 'Active'. Power edge-node-01 back on and verify it rejoins the cluster.

Step 10

Review Edge cluster capacity. NSX Manager → System → Fabric → Edge Clusters → edge-cluster-prod → Overview shows: Member count, Services hosted (T0, T1 with services, load balancers), Deployment status. For capacity planning: each Edge node can host a limited number of T1 service routers — Large form factor supports up to 400 T1 service instances. Monitor: get edge-cluster status from any Edge node CLI.

Validation Gate

Check: Edge cluster operational with DPDK fastpath and validated throughput

Expected: Edge cluster formed with 2+ nodes, dataplane service running and fastpath interfaces up, anti-affinity rules enforced, baseline throughput recorded and compared with the Large form factor sizing band (2-10 Gbps total), Edge failure/recovery validated with sub-second failover (BFD-enabled)

Common Errors

Edge VM deployment fails with 'Insufficient resources'
Fix: Edge VMs require 100% CPU and memory reservation. Verify the target host has unreserved capacity. For Large form factor: 8 vCPU + 32 GB free on a single NUMA node. Check: vSphere Client → Host → Monitor → Hardware → NUMA Topology.
Edge shows 'Controller Connectivity: Down'
Fix: Edge VM cannot reach NSX Controllers. Check: (1) Edge management IP is correct and on the right VLAN, (2) DNS resolves NSX Manager FQDN from Edge, (3) no firewall blocking port 1235 (Edge-to-Controller). SSH to Edge and run: get managers — should show connected status.
Edge forwards nothing although it shows Up: dataplane service down
Fix: Run get service dataplane on the Edge; if the state is stopped, start service dataplane (temporary workaround) and collect a support bundle — the NSX 9.0 docs tie this to memory leaks in Edge processes. Also check the Edge VM has 100 % CPU and memory reservation and that the Edge hosts' host switch runs Enhanced Data Path.
iperf3 throughput significantly lower than expected
Fix: Check: (1) Edge uplink speed (10GbE vs 25GbE physical NIC), (2) iperf3 using enough parallel flows (-P 8 minimum for ECMP), (3) NUMA misalignment causing cross-NUMA memory access penalties, (4) physical switch port speed/duplex settings.

Final Validation

NSX Edge cluster deployed with DPDK-accelerated data path and validated performance

✓ Edge node health → All Edge nodes: Configuration=Success, Status=Up, Manager+Controller=Connected

✓ DPDK datapath → get service dataplane shows running and fp-eth0/fp-eth1/fp-eth2 are up on all Edge nodes

✓ Edge cluster formation → Cluster shows correct member count with allocation rules configured

✓ Anti-affinity rules → DRS rule prevents Edge VMs on same host

✓ Throughput baseline → Baseline recorded; a Large Edge is sized for 2-10 Gbps total throughput, so plan Extra Large or more Active-Active (ECMP) Edge nodes above that

✓ Failover validation → Traffic recovers within BFD timeout after Edge node failure

Cleanup / Restore

• Remove T0/T1 gateways from Edge cluster (if created in this lab)

• Delete Edge cluster (System → Fabric → Edge Clusters → Delete)

• Delete Edge transport nodes (System → Fabric → Edge Transport Nodes → Delete — this powers off and removes Edge VMs)

• Remove anti-affinity DRS rules from vSphere

Design Reflection (VCDX)

Edge cluster design is a high-value VCDX topic. Key decisions: (1) Form factor — Small is for proof-of-concept only; Medium suits L2-L4 services below 2 Gbps total throughput; Large suits 2-10 Gbps and L7 load balancing; Extra Large is for higher throughput (VCF 9.0.1 docs p.2043). (2) Placement — dedicated Edge hosts vs shared, NUMA alignment, anti-affinity rules. (3) Cluster sizing — N+1 minimum; N+2 for environments requiring zero-downtime maintenance.
(4) Fastpath resourcing — every Edge VM already uses DPDK fastpath interfaces, so the design levers are form factor, 100 % CPU/memory reservation, NUMA alignment and Enhanced Data Path on the host switch. (5) Edge-per-rack vs centralized — affects failure domain and latency. (6) In VCF 9.0, Edge clusters are deployed from the vSphere Client (Networks → Network Connectivity → Configure Network Connectivity; VCF 9.0.1 docs p.1576) — understanding the NSX Manager process helps troubleshoot that workflow.

Requirements

  • 10+ Gbps north-south throughput per T0 gateway
  • Sub-second failover for stateful services
  • Support for 400+ T1 service instances per Edge cluster

Constraints

  • Large Edge VM requires 8 vCPU + 32 GB with 100% reservation per Edge node
  • DPDK cores are consumed even at idle — cannot overcommit Edge host CPU
  • Maximum 10 Edge nodes per cluster (practical limit based on control plane scalability)

Assumptions

  • Dedicated hosts for Edge VMs (no workload contention)
  • Physical uplinks are 10GbE minimum (25GbE recommended for DPDK throughput)
  • BFD is enabled for all BGP neighbors to achieve sub-second failover

Risks

  • Undersized Edge form factor causes packet drops under load — difficult to diagnose without DPDK CPU stats
  • Single Edge cluster for all T0/T1 creates a blast radius — consider per-domain Edge clusters for isolation
  • NUMA misalignment silently degrades throughput by 30-40% — always verify NUMA placement post-deployment

Self-Assessment Discussion Prompts

  1. How would you design Edge placement for a 3-site VCF deployment with stretched clusters?
  2. What is the impact of running Edge VMs on the same hosts as production workloads?
  3. How do you calculate the required number of Edge nodes for a given north-south bandwidth requirement?
  4. When would you choose Active-Standby over Active-Active for a Tier-0 gateway, and how does that affect Edge sizing?

Extensions

Deploy Extra Large Edge nodes and benchmark throughput improvement vs Large

Configure multi-TEP on Edge nodes and verify ECMP across multiple uplinks

Test Edge maintenance mode — migrate services to partner Edge and validate zero-downtime

Configure Edge node syslog forwarding and SNMP monitoring for production observability

⚠ Known Pitfalls

Deploying Small form factor Edge for production — Small is for proof-of-concept only and L7 rules (firewall, load balancing) are not realized on a Tier-1 gateway behind it
Ignoring NUMA alignment — DPDK performance degrades 30-40% with cross-NUMA memory access; verify with esxtop (NUMA home node column)
Placing all Edge nodes on the same host — a single host failure takes down the entire Edge cluster; anti-affinity is mandatory
Not reserving 100% CPU/memory for Edge VMs — contention from other VMs causes packet drops in the DPDK poll-mode driver

References

  • VCF 9.0 documentation — NSX Edge VM System Requirements and NSX Edge VM Resource Requirements (VCF 9.0.1 docs p.2042-2043): https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-9-0-and-later/9-0.html
  • VCF 9.0 documentation — Configure Network Connectivity (Centralized Connectivity), Edge cluster deployment from vCenter (VCF 9.0.1 docs p.1576-1577): https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-9-0-and-later/9-0.html
  • VCF 9.0 documentation — Enhanced Data Path (VCF 9.0.1 docs p.1965): https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-9-0-and-later/9-0.html
Was this page useful?
Type to search. ↑ ↓ to move, Enter to open, Esc to close.