Maximizing VMware Performance and CPU Utilization

June 14, 2019 | Susan Bilder

In a previous post we discussed overcommitting VMware host memory – the same can be done with host CPU. As per the Performance Best Practices for VMware vSphere 6.7:

In most environments ESXi allows significant levels of CPU overcommitment (that is, running more vCPUs on a host than the total number of physical processor cores in that host) without impacting virtual machine performance. (P. 20, ESXi CPU Considerations)

Optimize VMware Performance and CPU Utilization

This post will discuss calculating CPU resources, considerations in assigning resources to virtual machines (VMs), and which metrics to monitor to ensure CPU overcommitment does not affect VM performance.

Our Overcommitting VMware Resources Whitepaper delivers the guidelines you need to ensure that you are properly allocating your host resources without sacrificing performance.


Calculating available Host CPU Resources

The number of physical cores (pCPU) available on a host is calculated as:

(# Processor Sockets) X (# Cores/Processor)  = # Physical Processors (pCPU)

If the cores use hyperthreading, the number of logical cores is calculated as:

(# pCPU) X (2 threads/physical processor) = # Virtual Processors (vCPU)

For example, if you have 2 processors with 6 cores each:

(2 Processor Sockets) X (6 Cores/Processor)  = 12 Physical Processors (pCPU)
(12 pCPU) X (2 threads/physical processor) = 24 Virtual Processors (vCPU)

Please note that hyperthreading does not actually double the available pCPU. Hyperthreading works by providing a second execution thread to a processor core. When one thread is idle or waiting, the other thread can execute instructions. This can increase efficiency if there is enough CPU Idle time to provide for scheduling two threads, but in practice performance increases are up to a maximum of 30% and are strongly application dependent.

Considerations in Allocating vCPUs to VMs

► Best Practices recommendations

  • Start with one vCPU per VM and increase as needed.
  • Do not allocate more vCPUs than needed to a VM as this can unnecessarily limit resource availability for other VMs and increase CPU Ready wait time.
  • The exact amount of CPU overcommitment a VMware host can accommodate will depend on the VMs and the applications they are running. A general guide for performance of {allocated vCPUs}:{total vCPU} from the Best Practices recommendations is:
    • 1:1 to 3:1 is no problem
    • 3:1 to 5:1 may begin to cause performance degradation
    • 6:1 or greater is often going to cause a problem

► Non-Uniform Memory Architecture (NUMA)

In a previous post on minimizing CPU latency with NUMA, we discussed performance degradation on multiprocessor VMs in which the number of vCPUs was greater than the number of vCPUs in a NUMA node.

Generally, try to keep multiprocessor VMs sized to fit within a NUMA node.

► Co-stop

VMware schedules all the vCPUs in a VM at the same time. If all the allocated vCPUs are not available at the same time, then the VM will be in a state of "co-stop" until the host can co-schedule all vCPUs.

In its simplest form co-stop indicates the amount time after the first vCPU is available until the remaining vCPUs are available for the VM to run.

Sizing the VMs to use the least number of vCPU’s possible minimizes the time needed for co-stop waits.

Effective & Affordable: Try Longitude Today

Monitor for VMware problems and ensure that your virtual infrastructure performs optimally. Use the Capacity Planner to avoid virtual machine sprawl, over-provisioning, and excess use of resources.


Monitoring VMware CPU metrics

Monitor the following metrics to fine tune the number of vCPUs allocated per VM and to ensure that CPU overcommitment does not degrade performance:

► VM CPU Utilization

Monitor CPU Utilization by the VM to determine if additional vCPUs are required or if too many have been allocated. CPU use can be monitored through VMware or through the VM’s operating system.

Utilization should generally be <= 80% on average, and > 90% should trigger an alert, but this will vary depending on the applications running in the VM.

► VM CPU Ready

VM CPU Ready is a measure of the time a VM has to wait for CPU resources from the host.

VMware recommends CPU ready should be less than 5%.

Longitude CPU Ready

Figure 1: Longitude Report of VMware showing high CPU Ready values.

► Co-Stop

Applicable to VMs with multiple vCPUs – is a measure of the amount time after the first vCPU is available until the remaining vCPUs are available for the VM to run.

A co-stop percent that is persistently >3% is an indication that a right-sizing exercise may be in order.

esxtop showing high co-stop

Figure 2: esxtop showing a VM with a high co-stop value.

► VMware host CPU Utilization

Monitor CPU Utilization on the VM host to determine if CPU use by the VMs is approaching the maximum CPU capacity.

As with CPU usage on VMs, CPU utilization at 80% - 90% should be considered a warning level, and >= 90% indicates that the CPUs are approaching an overloaded condition.

Longitude Capacity Planner
Figure 3: Longitude Capacity Planner showing host CPU approaching capacity.


Overcommitting CPU allows you to maximize use of host CPU resources, but make sure to monitor overcommitted hosts for CPU use, and CPU Ready and Co-stop percentages.

Avoid oversizing VMs with more vCPU’s than needed. Consider NUMA architecture and the effect of co-stop waits when creating VMs with multiple vCPUs.

Want to learn more?

Download our Overcommitting VMware Resources Whitepaper for the guidelines you need to ensure that you are getting the most out of your host resources without sacrificing performance.

New call-to-action


Editor's Note:  This post was originally published in February 2017 and has been updated for freshness accuracy, and comprehensiveness.

We value your privacy and will not display or share your email address

Sign Up for the Blog

Heroix will never sell or redistribute your email address.