vSphere cluster groups are sets of ESXi hosts that have a common compute entity; there are from 2 to 64 hosts per vSphere cluster when vSphere HA and DRS are activated at the cluster level. Resource pools are created under a vSphere cluster instance, and vCenter is able to manage multiple vSphere clusters instances, as there is no hard limit to the number of vSphere clusters. We can create different types of vSphere clusters, such as management clusters, compute clusters, and Edge clusters, since PKS fully leverages the vSphere cluster construct.

The following vSphere clusters are recommended in a typical PKS deployment:

• Management cluster:
  • Hosted components: vCenter, NSX manager, and controller VMs
  • vSphere HA and DRS enabled
  • ESXi hosts need to be NSX prepared, as micro-segmentation is enforced on the hosted VMs
• Compute cluster(s):
  • Hosted components: Kubernetes (K8s) clusters nodes VMs
  • vSphere HA and DRS should be enabled, as BOSH will check whether DRS is turned on
  • ESXi hosts need to be NSX prepared
• Edge cluster:
  • Hosted components: NSX Edge Nodes VMs
  • vSphere HA and DRS enabled
  • ESXi hosts don't need to be NSX prepared

The PKS Management Plane can reside on the management cluster or compute cluster, depending on the selected design scenario. PKS Management Plane VMs are the Ops Manager, BOSH, the PKS control plane, and Harbor. 

The PKS data plane (or compute plane) will only reside in a compute cluster. Up to three K8s master nodes and 50 worker nodes are allowed per K8s cluster, and many K8s clusters can be created in the same PKS environment.

The K8s master node also hosts the etcd component. vSphere DRS and HA must be enabled on the vSphere compute cluster. vSphere DRS Automation has to be set to Partially Automated or Fully Automated. vSphere HA is set with Host failure = Restart VMs.

The following are the compute and storage requirements for the PKS component: