“Overcommitting RAM on nested ESXi hosts leads to silent hypervisor swapping and disk latency spikes. Proper NUMA node mapping prevents performance degradation.”

When running 5 concurrent automated SDDC test pods on a physical server host (Dual Intel Xeon, 320GB RAM, 20 Cores) at NTT Data, nested vCenter instances experienced 100% CPU wait times.

Root Cause: NUMA Node Boundary Crossing & Memory Swapping

Each physical CPU socket governed 160GB of RAM across a local NUMA node. Allocating a single nested vCenter VM with 32GB RAM spanning across both physical NUMA nodes introduced high inter-socket QPI bus latency.

We restructured our Ansible deployment sizing profiles (answerfile.yml) to enforce strict NUMA Affinity Alignment and thin disk provisioning.

[!NOTE] Keep individual nested VM sizing within the boundary of a single physical NUMA socket (e.g. max 160GB RAM and 10 cores per VM) to eliminate socket-hopping latency.

# # Sizing Profile from answerfile.yml
sddc_sizing_profile:
  vcenter:
    vcpus: 4
    ram_mb: 16384  # Aligned within 1 NUMA node
    disk_mode: "thin"
  esxi_nested:
    vcpus: 4
    ram_mb: 32768  # 32GB per nested host
    count: 3

The Verdict

Key Takeaway

Align Nested SDDC Workload Allocations to Physical NUMA Boundaries.

Size nested virtual machines within single physical NUMA nodes and utilize thin provisioning to maximize pod density without triggering memory ballooning.