Academy/vSphere Foundation 9.0 Administrator (2V0-16.25)/Lab: Deploy a VKS Cluster with vSphere Kubernetes Service (VKS)
This lab targets VCF 9.0

Lab: Deploy a VKS Cluster with vSphere Kubernetes Service (VKS)

VCF 9.0Advancedadminkubernetes⏱ 120 min

VKS (formerly TKG on Supervisor). Uses Cluster API. Antrea CNI in VVF (no NSX NCP). Requires Supervisor enabled (Lab 10).

Objectives

  • Create a VKS cluster manifest using Cluster API topology (ClusterClass)
  • Deploy a managed Kubernetes cluster via kubectl apply
  • Authenticate to TKG cluster and deploy a sample workload
  • Understand VKS architecture: Cluster API, Antrea CNI, vSAN CSI
  • Perform cluster lifecycle operations: scale workers, upgrade K8s version

Prerequisites

Supervisor enabled on VVF cluster (Lab 10 completed). Namespace 'dev-team' with resource quotas. Minimum 48GB additional RAM available for TKG cluster (3 control + 3 worker nodes).

Prior labs: vvf-admin-10

Required skills:

  • Kubernetes administration (kubectl, deployments, services)
  • YAML manifest authoring
  • Container concepts (images, registries, pods)

Lab Environment

Holodeck VVF pod with Supervisor running. TKG cluster deployed as set of VMs managed by Cluster API.

Credentials

SystemUsernamePassword
Supervisoradministrator@vsphere.localSame as vCenter
TKG Clusteradministrator@vsphere.localSame as vCenter

Tasks

Task 1 Create and Deploy TKG Cluster

manageability

VKS uses Cluster API to provision managed Kubernetes clusters. Understanding the manifest structure, version selection, and deployment process is exam-critical for Obj 4.4.

Step 1

Authenticate to Supervisor:
kubectl vsphere login --server=<supervisor-api-ip> --vsphere-username=administrator@vsphere.local --insecure-skip-tls-verify
kubectl config use-context dev-team

Step 2

List available Kubernetes releases:
kubectl get tanzukubernetesreleases

Note the READY VKr versions the Supervisor offers. VCF 9.0 bundles VKS 3.3.1, which supports VKr 1.28 and later (1.32 added) and drops compatibility for 1.27.x (VCF 9.0.1 docs p.107) — so a 1.27.x version will not be offered.

List of available VKr versions (1.28+) with READY=True status.
Step 3

Create the VKS cluster manifest (vks-cluster.yaml). Use the builtin-generic ClusterClass that matches your VKS version (the tanzukubernetescluster class is deprecated in VCF 9.0; the docs use builtin-generic-v3.x.x) and a READY VKr from step 2:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: lab-cluster-01
  namespace: dev-team
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
      - 10.244.0.0/16
    services:
      cidrBlocks:
      - 10.96.0.0/16
  topology:
    class: builtin-generic-v3.3.0   # match the ClusterClass shipped with your VKS version
    version: <a READY VKr from step 2, e.g. v1.32.x+vmware.x-fips>
    controlPlane:
      replicas: 1
    workers:
      machineDeployments:
      - class: node-pool
        name: worker-pool-1
        replicas: 2
    variables:
    - name: vmClass
      value: best-effort-small
    - name: storageClass
      value: vsan-default-storage-policy

Check the variable names against the ClusterClass (kubectl get clusterclass builtin-generic-v3.3.0 -o yaml) before applying.

For the lab, use 1 control plane + 2 workers to conserve resources. Production: 3 control plane + 3+ workers. The fenced block keeps the YAML indentation intact when the page renders.
Step 4

Deploy the cluster:
kubectl apply -f vks-cluster.yaml

cluster.cluster.x-k8s.io/lab-cluster-01 created
Step 5

Monitor deployment progress (takes 15-25 minutes):
kubectl get cluster lab-cluster-01 -n dev-team -w

Also monitor in vCenter: VMs named 'lab-cluster-01-*' will be created.

Cluster phases: Provisioning → Provisioned. VMs appear in vCenter inventory under dev-team resource pool.
Step 6

Verify cluster is ready:
kubectl get cluster lab-cluster-01 -n dev-team

PHASE=Provisioned, READY=True

Validation Gate

Check: kubectl get cluster lab-cluster-01 -n dev-team

Expected: PHASE=Provisioned, 1 control plane + 2 worker VMs running in vCenter

Common Errors

Cluster stuck in 'Provisioning' for >30 minutes
Cause: Insufficient resources in namespace quota or TKR image not downloaded
Fix: kubectl describe cluster lab-cluster-01 — check Events. Verify namespace quota has enough CPU/memory. Check TKR availability: kubectl get tkr.
'no matches for kind Cluster' error
Cause: Cluster API CRDs not installed on Supervisor
Fix: Verify Supervisor is healthy: vCenter → Workload Management. The VKS Supervisor Service must be installed and running on the Supervisor.
Worker nodes not joining cluster
Cause: Antrea CNI pod network issue, or the Supervisor load balancer not assigning VIPs
Fix: kubectl describe svc and the cluster events; check the load balancer's VIP pool (Foundation Load Balancer or Avi). Verify pod CIDR (10.244.0.0/16) doesn't conflict with existing networks.

Task 2 Authenticate to TKG Cluster and Deploy Workload

manageability

Authenticating to the TKG cluster and deploying a workload validates the full developer experience — from cluster provisioning to application deployment.

Step 1

Login to TKG cluster:
kubectl vsphere login --server=<supervisor-api-ip> --vsphere-username=administrator@vsphere.local --tanzu-kubernetes-cluster-name=lab-cluster-01 --tanzu-kubernetes-cluster-namespace=dev-team --insecure-skip-tls-verify

Step 2

Switch context:
kubectl config use-context lab-cluster-01

Step 3

Verify cluster nodes:
kubectl get nodes

1 control-plane node + 2 worker nodes, all in Ready status.
Step 4

Check cluster components:
kubectl get pods -A

System pods running: antrea-agent (on each node), antrea-controller, coredns, etcd, kube-apiserver, kube-controller-manager, kube-scheduler.
Step 5

Deploy sample application:
kubectl create deployment nginx --image=nginx:latest --replicas=2
kubectl expose deployment nginx --port=80 --type=LoadBalancer

deployment.apps/nginx created
service/nginx exposed
Step 6

Wait for the LoadBalancer IP (the Supervisor load balancer — Foundation Load Balancer or Avi — assigns the VIP):
kubectl get svc nginx -w

EXTERNAL-IP assigned from the Supervisor load balancer's VIP range (Foundation Load Balancer or Avi in VCF 9.0; HAProxy is deprecated in 9.0, VCF 9.0.1 docs p.240). Use the range you configured in Lab 10.
If EXTERNAL-IP stays '<pending>', check the Supervisor load balancer's VIP pool (Foundation Load Balancer or Avi) for free addresses and connectivity.
Step 7

Test application:
curl http://<EXTERNAL-IP>

nginx default welcome page HTML returned.

Validation Gate

Check: curl http://<EXTERNAL-IP> returns nginx welcome page

Expected: HTTP 200 with nginx default page content

Task 3 Scale Workers and Perform Cluster Lifecycle Operations

availability

Day-2 operations — scaling and upgrading — are key exam topics. Understanding how Cluster API handles rolling updates is critical.

Step 1

Scale worker pool from 2 to 3:
kubectl patch cluster lab-cluster-01 -n dev-team --type merge -p '{"spec":{"topology":{"workers":{"machineDeployments":[{"class":"node-pool","name":"worker-pool-1","replicas":3}]}}}}'

cluster.cluster.x-k8s.io/lab-cluster-01 patched
Step 2

Monitor scale-out (switch back to Supervisor context first):
kubectl config use-context dev-team
kubectl get machines -n dev-team -w

New Machine object created → VM provisioned → node joins cluster. Takes 3-5 minutes.
Step 3

Verify 3 workers:
kubectl config use-context lab-cluster-01
kubectl get nodes

1 control plane + 3 workers, all Ready.
Step 4

Scale back to 2 workers:
kubectl config use-context dev-team
kubectl patch cluster lab-cluster-01 -n dev-team --type merge -p '{"spec":{"topology":{"workers":{"machineDeployments":[{"class":"node-pool","name":"worker-pool-1","replicas":2}]}}}}'

One worker node cordoned, drained, and deleted. Workloads rescheduled to remaining nodes.
Step 5

Verify nginx pods redistributed:
kubectl config use-context lab-cluster-01
kubectl get pods -o wide

2 nginx pods running on remaining 2 worker nodes.
Step 6

Upgrade the cluster to the next available VKr (switch to the Supervisor context first):
kubectl config use-context dev-team
kubectl get tanzukubernetesreleases # pick the next READY version above the current one
kubectl patch cluster lab-cluster-01 -n dev-team --type merge -p '{"spec":{"topology":{"version":"<next READY VKr>"}}}'
kubectl get machines -n dev-team -w

VKS clusters use a rolling update model: new control plane machines replace old ones one at a time, then the workers (VCF 9.0.1 docs p.5743). Changing spec.topology.version to a newer VKr initiates the rolling update; the docs do it with kubectl edit cluster/<name>, and the patch above changes the same field. While it runs the cluster's TopologyReconciled condition shows False ('Update a v1beta1/v1beta2 VKS Cluster by Editing the VKr Version', vSphere Supervisor 9.0 Services and Standalone Components).

cluster.cluster.x-k8s.io/lab-cluster-01 patched; machines replaced one at a time; kubectl get nodes (cluster context) shows the new version on every node

Validation Gate

Check: kubectl get nodes shows 1 control plane + 2 workers, kubectl get pods shows nginx pods running

Expected: Cluster scaled back to 2 workers, all pods healthy

Final Validation

VKS cluster deployed, workload running with LoadBalancer, scaling and a rolling VKr upgrade demonstrated.

✓ TKG cluster lab-cluster-01 in Provisioned state → kubectl get cluster shows READY=True

✓ nginx deployment accessible via LoadBalancer VIP → curl returns nginx welcome page

✓ Cluster scale-out and scale-in completed successfully → Worker count changed 2→3→2 without workload disruption

✓ Cluster upgraded to the next VKr → All nodes report the new Kubernetes version; nginx stayed available during the rolling update

Cleanup / Restore

Snapshot: post-vks-lab

• Delete nginx deployment: kubectl delete deployment nginx && kubectl delete svc nginx

• Optionally delete TKG cluster: kubectl config use-context dev-team && kubectl delete cluster lab-cluster-01 (takes 10-15 min)

• Take snapshot 'post-vks-lab'

Design Reflection (VCDX)

Panelist: What's the blast radius if Supervisor fails — does the TKG cluster keep running? How do you handle K8s version upgrades across multiple TKG clusters? What's your persistent storage strategy for stateful workloads on VKS?

Requirements

  • Managed Kubernetes clusters for developer teams
  • Self-service cluster provisioning
  • Automated scaling for variable workloads
  • Persistent storage for stateful applications

Constraints

  • Antrea CNI in VVF (no NSX network policies beyond basic K8s NetworkPolicy)
  • The Supervisor load balancer VIP range (Foundation Load Balancer or Avi) limits the number of LoadBalancer services
  • TKG cluster resources consume namespace quota
  • K8s version tied to TKR availability (VMware release cadence)

Assumptions

  • Developers manage their own TKG clusters within namespace boundaries
  • Load balancer VIP range sufficient for all LoadBalancer services
  • Container images accessible from TKG nodes (registry connectivity)

Risks

  • TKG cluster upgrade failure can leave cluster in mixed-version state
  • Supervisor failure doesn't affect running TKG clusters but blocks management operations
  • Resource exhaustion if multiple large TKG clusters share namespace
  • Antrea CNI network policies less mature than NSX NCP

Self-Assessment Discussion Prompts

  1. How does VKS handle K8s version upgrades — in-place or blue-green?
  2. What monitoring should you add for TKG cluster health beyond Supervisor status?
  3. How would you design multi-tenancy with VKS — one cluster per team or shared cluster with namespace isolation?

References

Was this page useful?
Type to search. ↑ ↓ to move, Enter to open, Esc to close.