Academy/NSX 4.x Network Virtualization Professional (2V0-41.24)/Lab: Create Overlay Segment & Verify Tunnel Endpoints
This lab targets VCF 9.0

Lab: Create Overlay Segment & Verify Tunnel Endpoints

VCF 9.0Intermediatevcp-foundation⏱ 90 min

NSX 9.0.x (feature set inherited from the NSX 4.2 line; NSX 4.2 docs remain a valid technical reference) overlay networking — GENEVE encapsulation (UDP 6081), N-VDS integration, TEP validation

Objectives

  • Create an overlay segment on an NSX overlay transport zone
  • Understand GENEVE encapsulation and VNI allocation
  • Verify Tunnel Endpoint (TEP) establishment between ESXi hosts
  • Validate VM-to-VM connectivity across the overlay fabric
  • Inspect N-VDS forwarding tables and ARP suppression

Prerequisites

VCF 9.0 management domain operational, NSX Manager deployed, ESXi hosts prepared as transport nodes with overlay transport zone membership and TEP IP pools configured

Required skills:

  • NSX Manager UI navigation
  • SSH access to ESXi hosts
  • Basic GENEVE/overlay networking concepts

Lab Environment

VCF 9.0 management domain with minimum 2 ESXi hosts configured as NSX transport nodes. Each host has vmk10 (TEP vmkernel) on a dedicated VLAN-backed TEP segment. NSX Manager cluster (3-node or single for lab) operational.

Tasks

Task 1 Create Overlay Segment and Verify TEP Connectivity

Production overlay designs must consider TEP VLAN sizing (jumbo frames required — MTU 1700+ for GENEVE overhead), TEP IP pool exhaustion, and multi-TEP for bandwidth scaling. Always use IP pools (not DHCP) for TEP addressing in VCF deployments.

Build an overlay segment from scratch and validate the entire forwarding path — from NSX Manager control plane down to the ESXi kernel data plane — ensuring GENEVE tunnels are established, VNI is allocated, and VM-to-VM traffic traverses the overlay correctly.

Step 1
Pre-flight: Verify transport node status. In NSX Manager → System → Fabric → Transport Nodes, confirm all ESXi hosts show 'Success' for Configuration State and 'Up' for Node Status. If any host shows 'In Progress' or 'Failed', resolve before proceeding (common cause: TEP vmkernel port missing or MTU mismatch).
Step 2

Verify TEP connectivity between hosts. SSH to esxi-01, run: vmkping ++netstack=vxlan -d -s 8972 <esxi-02-TEP-IP>. The -d flag sets DF bit, -s 8972 tests jumbo frame path. Success confirms the underlay supports GENEVE encapsulation overhead (50-byte header). If this fails, check physical switch MTU (must be ≥ 9000) and TEP VLAN configuration.

Step 3
Create overlay segment. NSX Manager → Networking → Segments → Add Segment. Configure: Name='prod-web-seg', Connected Gateway=None (L2-only for now), Transport Zone=select your overlay TZ (e.g., 'tz-overlay'), Subnets=leave empty (no gateway yet), Admin State=Up. Click Save → the segment receives an auto-allocated VNI (e.g., 72001).
Step 4
Verify segment realization. After saving, click the segment name → check Realization status = 'Success'. Navigate to Related Resources → Transport Nodes tab — all hosts in the overlay TZ should show the segment as realized. Note the VNI value from the Overview tab — you will verify this on the host.
Step 5
Verify on ESXi host via CLI. SSH to esxi-01, run: nsxcli → get logical-switches. Confirm your segment appears with the correct VNI. Then run: get logical-switch <VNI> arp-table and get logical-switch <VNI> mac-table — both should be empty (no VMs attached yet). Run: get logical-switch <VNI> vtep-table — should list TEP IPs of all other transport nodes in the TZ.
Step 6
Attach test VMs. In vSphere Client, edit VM 'test-web-01' on esxi-01 → Network Adapter 1 → Browse → select 'prod-web-seg'. Repeat for 'test-web-02' on esxi-02. Power on both VMs and assign IPs: test-web-01=10.10.1.11/24, test-web-02=10.10.1.12/24 (no gateway needed for this L2 test).
Step 7
Validate overlay connectivity. From test-web-01 console: ping 10.10.1.12. Traffic flows: VM → vNIC → VSIP filter (DFW) → N-VDS dvPort → GENEVE encapsulation (src TEP → dst TEP, UDP 6081, VNI header) → physical underlay → destination host reverse path. Verify the ping succeeds.
Step 8
Inspect forwarding tables post-traffic. SSH to esxi-01, run: nsxcli → get logical-switch <VNI> mac-table — should now show test-web-02's MAC with its TEP IP as the destination tunnel endpoint. Run: get logical-switch <VNI> arp-table — should show 10.10.1.12 → MAC mapping (ARP suppression cache). These tables prove the overlay data plane is fully converged.
Step 9

Packet capture (optional but valuable). On esxi-01, capture GENEVE-encapsulated traffic: pktcap-uw --uplink vmnic0 --capture UplinkSndKernel -o /tmp/geneve.pcap. Transfer the pcap and open in Wireshark — filter on 'udp.port==6081'. You should see the outer IP header (TEP-to-TEP), GENEVE header with VNI, and inner Ethernet frame (VM-to-VM). This is exam-critical knowledge.

Step 10
Segment profile exploration. In NSX Manager → Networking → Segments → Segment Profiles, review the default profiles: Segment Security (BPDU filter, DHCP snooping), MAC Discovery (MAC learning, unknown unicast flooding), IP Discovery (ARP snooping, ND snooping), SpoofGuard (disabled by default). For production, enable SpoofGuard and ARP snooping at minimum. Note: VCF applies these defaults automatically.

Validation Gate

Check: Complete overlay data path verification

Expected: Overlay segment created with auto-allocated VNI, TEP tunnels UP between all hosts (jumbo frame path verified), test VMs communicate across hosts via GENEVE encapsulation, MAC/ARP tables populated correctly on both hosts

Common Errors

TEP vmkping fails with jumbo frames
Fix: Physical switch MTU must be ≥ 9000 on all ports carrying TEP VLAN. Check: esxcfg-nics -l (host NIC MTU), then verify each switch hop in the physical path.
Segment shows 'Partial Success' in realization
Fix: One or more transport nodes failed to realize the segment. Check System → Fabric → Transport Nodes for the failing host — common causes: host disconnected from NSX Manager, or NSX VIB installation incomplete.
VM connectivity works on same host but fails cross-host
Fix: TEP routing issue — ensure TEP IPs on different hosts can reach each other via the underlay. If TEPs are on different subnets, a Layer 3 route must exist between TEP VLANs.
ARP table not populating despite successful pings
Fix: ARP suppression may be disabled on the segment profile. Verify IP Discovery profile has ARP Snooping enabled. Without it, ARP floods as BUM traffic — functional but inefficient.

Final Validation

Overlay segment fully operational with verified TEP tunnels and cross-host VM connectivity

✓ Segment realization status → Success on all transport nodes in the overlay TZ

✓ TEP tunnel health → All TEP-to-TEP tunnels show BFD status UP (get tunnel-ports)

✓ VM-to-VM ping across hosts → 0% packet loss on overlay segment

✓ MAC/ARP tables populated → Remote VM entries present with correct TEP destination

✓ GENEVE encapsulation verified → Packet capture shows UDP 6081 with correct VNI in header

Cleanup / Restore

• Disconnect test VMs from segment (revert to original network)

• Delete 'prod-web-seg' segment from NSX Manager (Networking → Segments → Delete)

• Verify segment removed from hosts: nsxcli → get logical-switches

Design Reflection (VCDX)

In a VCDX defense, be prepared to justify overlay vs. VLAN segment choices. Key arguments: overlay provides workload mobility without L2 stretching at the physical layer, supports micro-segmentation natively, and scales beyond 4096 VLAN limit (16M VNIs). Discuss TEP design — single vs. multi-TEP per host, TEP VLAN sizing, and failure domain isolation (separate TEP pools per rack for blast radius containment).

Requirements

  • East-west VM communication across hosts without VLAN trunking changes
  • Support for >4096 logical networks
  • Workload mobility without physical network reconfiguration

Constraints

  • Physical underlay must support MTU ≥ 9000 for GENEVE overhead
  • TEP IP pools must be pre-provisioned (VCF does not support DHCP for TEPs)
  • Maximum 16 TEPs per host (practical limit for bandwidth scaling)

Assumptions

  • All physical switches in the TEP path support jumbo frames
  • TEP VLAN is a dedicated, low-latency network (not shared with vMotion)
  • NSX Manager cluster is healthy and all hosts are connected

Risks

  • TEP MTU mismatch causes silent packet drops — hard to diagnose without pktcap-uw
  • TEP IP pool exhaustion during scale-out prevents new hosts from joining overlay
  • Single TEP VLAN creates a network failure domain — consider multi-VLAN TEP design for resilience

Self-Assessment Discussion Prompts

  1. Why does VCF mandate IP pools over DHCP for TEP addressing?
  2. How would you design TEP networks for a 64-host cluster across 4 racks?
  3. What is the impact of enabling ARP suppression vs. allowing ARP flooding?
  4. When would you choose head replication vs. hierarchical replication for BUM traffic?

Extensions

Configure multi-TEP (2 TEPs per host) and verify ECMP load balancing across both uplinks

Create a VLAN-backed segment on the same host and compare forwarding behavior (no GENEVE encap)

Test BUM traffic handling: send a broadcast from one VM, capture on the uplink of all hosts to observe head replication

⚠ Known Pitfalls (from Community KB)

Using MTU 1500 on physical underlay — GENEVE adds 50 bytes overhead; packets >1450 bytes get dropped silently
Forgetting to check TEP vmkernel stack — TEP traffic uses the 'vxlan' netstack, not the default stack; regular vmkping won't test the right path
Creating segments on wrong transport zone — overlay segments on VLAN TZ (or vice versa) will fail realization

References

  • NSX 4.2 Administration Guide — Segments: techdocs.broadcom.com
  • VCF 9.0 Networking Guide — Transport Zone Design: techdocs.broadcom.com
  • KB 93154 — Troubleshooting TEP Connectivity Issues
Was this page useful?
Type to search. ↑ ↓ to move, Enter to open, Esc to close.