Lab NN3: Geneve Encapsulation Deep-Dive & MTU Validation

VCF 9.0Intermediatevcap-advanced⏱ 75 min

Objectives

Troubleshoot overlay MTU issues end-to-end from VM to transport node to physical underlay.

Prerequisites

VCF lab environment deployed and operational

Lab Environment

Standard VCF lab environment for Advanced VCF 9.0 Networking (NSX & Advanced Routing)

Tasks

Task 1 Lab NN3: Geneve Encapsulation Deep-Dive & MTU Validation

Troubleshoot overlay MTU issues end-to-end from VM to transport node to physical underlay.

Step 1

Verify underlay MTU ≥1700 on all TOR/spine switches.

Step 2

Validate host TEP VMkernel MTU at 1700.

Step 3

Test with VM-to-VM large-packet flow (jumbo).

Step 4

Intentionally reduce underlay MTU and observe fragmentation / path MTU discovery failure.

Step 5

Fix and verify; document MTU design rule for the customer.

Validation Gate

Check: Verify lab completion

Expected: Lab exercise completed successfully

Common Errors

Not verifying MTU end-to-end before enabling overlay transport

Fix: GENEVE adds 50 bytes overhead. With outer IP/UDP headers, total overhead is ~54 bytes. If any hop in the path has MTU < 1600 (for 1500-byte inner payload), packets fragment or drop. Test: vmkping -d -s 8972 from TEP to TEP verifies jumbo frames. For non-jumbo paths, reduce inner MTU to 1450.

Confusing TEP IP assignment methods (DHCP vs IP pool vs static)

Fix: VCF uses IP pool for TEP assignment (configured during bring-up). Manual static assignment causes IP conflicts. DHCP works but loses control over TEP IP predictability. Stick with SDDC Manager-managed IP pools for VCF environments.

Packet capture not filtering for GENEVE protocol correctly

Fix: Standard tcpdump shows GENEVE packets as UDP on port 6081. To see inner payload, use: pktcap-uw --uplink vmnic0 --proto 0x11 --dstport 6081 --capture. Alternatively, use NSX-T CLI: 'get capture interface <id> file <filename>' for decoded overlay traffic.

Final Validation

Lab completed successfully

✓ All steps completed → No errors observed

Cleanup / Restore

• Revert to snapshot if needed

Design Reflection (VCDX)

GENEVE encapsulation understanding shows deep networking knowledge. VCDX panelists test MTU awareness, TEP design, and overlay troubleshooting methodology.

⚠ Known Pitfalls (from Community KB)

Assuming MTU 1500 works for overlay — it does, but with 50-byte overhead reducing effective payload and risking fragmentation with large frames.

Not testing MTU end-to-end before go-live — MTU issues cause intermittent connectivity problems that are difficult to diagnose.

Mark complete