Rightsizing of workload is always a huge topic in virtualized enterprise environments. It starts with how to get users to order the right size of VM and then eventually leads to how to Apply Rightsizing Recommendations to existing vSphere VMs. During this blog post, I mainly talk about oversized VMs, which are 95% of the Rightsizing Recommendations I have seen in various environments. Undersized VMs are often quickly identified by the application owner, but unused resources are rarely returned.
When you get into the customer perspective, a bit more memory and vCPUs than required for the application is no bad thing. The customer or application owner just wants to make sure the single service runs as good as possible. But when you switch to the perspective of the private cloud operator, all there slightly overprovisioned VMs lead to a huge waste of costs. To be honest, often it is not only a little waste but often quite a lot per VM.
For the private cloud operator the additional costs are not only the server hardware:
- Rackspace, power consumption and cooling
- LAN and SAN Ports
- VMware vSphere / vRealize Licenses
- Backups Sofware License (often based on CPU Sockets)
- OS Licenses (often based on CPU Sockets)
- Application Licenses (often based on CPU Sockets)
- Operational overhead during maintenance and hardware lifecycle
Is the private cloud environment even a VMware vSphere Metro Storage Cluster with 50% reserved capacity, the waste is even doubled.
As I mentioned at the beginning, waste is usually not “with malice aforethought”, there are even some factors that encourage it:
- Unbalanced T-Shirt Sizes during the order process
- Need for more memory adds also vCPUs
- Need for additional vCPU increases also memory
- Intransparent costs
- Individual VM
- Whole business unit
- Missing lifecycle capabilities
- De-Provision VM
- Re-Configure resources
- See own inventory
With vRealize Operations and vRealize Automation, VMware offers two powerful tools to proactively eliminate cost waste and visualize the potential for optimization of the existing workload.
vRealize Operations Rightsizing Recommendations
Brandon Gordon has written a great blog post about Rightsizing VMs with vRealize Operations, I will just summarize some of his remarks and then jump to the PowerCLI function I have created to Apply Rightsizing Recommendations to VMware vSphere VMs.
I usually use two different representations of the existing Rightsizing Recommendations in vRealize Operations:
The Rightsizing Quickstart gives a great overview of the up- and downsizing potential of the individual clusters and datacenters. To dig deeper into the individual environment groups or custom groups I typically switch to the view for Oversized Virtual Machines. The view can also be cloned and additional metrics added, for example, costs.
Both representations are based on the same key metrics related to the vSphere VMs:
Capacity Analytics Generated|CPU|Recommended Size (MHz):
The recommended amount of CPU Usable Capacity in MHz needed to maintain a green state for the entire time between now and the Green Time Remaining Score Threshold set in the policy + 30 Days.
Capacity Analytics Generated|Memory|Recommended Size (KB):
The recommended amount of Memory Usable Capacity in KB needed to maintain a green state for the entire time between now and the Green Time Remaining Score Threshold set in the policy + 30 Days.
Summary|Is Oversized:
If a VM is detected to be oversized for at least one resource type (CPU or Memory), the value will be set to 1. Otherwise, the value will be 0.
Summary|Is Undersized:
If a VM is detected to be undersized for at least one resource type (CPU or Memory), the value will be set to 1. Otherwise, the value will be 0.
Summary|Oversized|Virtual CPUs:
The recommended number of vCPUs to remove from an oversized VM.
Summary|Oversized|Memory (KB):
The recommended amount of memory in KB to remove from an oversized VM.
Summary|Undersized|Virtual CPUs:
The recommended number of vCPUs to add to an undersized VM.
Summary|Undersized|Memory (KB):
The recommended amount of memory in KB to add to an undersized VM.
These metrics are also available in the “All Metrics” Tab and respective via PowerCLI and the REST API.
It is important to know that there is a cap on recommendations:
- Downsizing recommendations are capped at 50% of the current configuration
- Upsizing is capped at 100% of the current configuration (not more than doubled)
In the case you have Actions enabled for the vCenter, the “Set CPU Count and Memory for VM” Action will predefine the recommended sizing parameter if a recommendation is available.
But keep in mind, although the recommendations are very solid, vRealize Operations can’t know everything about the Application and Environment. Some corner cases where Rightsizing Recommendations completely mislead:
- Active/passive cluster on application level
- Rarely running build and testing processes
- Applications that reserve failover capacity, e.g. Container Host
Apply Rightsizing Recommendations with PowerCLI
As PowerCLI includes a Module for VMware vRealize Operations with great interoperability to the vSphere Modules, it’s quite easy to get the required Metrics in PowerShell. This Snippet generates a Report that shows the Up- and Downsizing opportunities of given vSphere VMs:
Get-Module -Name VMware* -ListAvailable | Import-Module -Force
Connect-VIServer vCenter01
Connect-OMServer -AuthSource "AD" -Server vRops01
$OMResources = Get-VM -Name test-* | Get-OMResource
$View = @()
foreach ($OMResource in $OMResources){
$DownSize = ($OMResource | Get-OMStat -Key "summary|oversized" -From ([DateTime]::Now).AddMinutes(-120) | Select-Object -Last 1).Value
$UpSize = ($OMResource | Get-OMStat -Key "summary|undersized" -From ([DateTime]::Now).AddMinutes(-120) | Select-Object -Last 1).Value
# Mem is in KB
if($DownSize -gt 0){
$DownSizeMem = ($OMResource | Get-OMStat -Key "summary|oversized|memory" -From ([DateTime]::Now).AddMinutes(-120) | Select-Object -Last 1).Value
$DownSizeCPU = ($OMResource | Get-OMStat -Key "summary|oversized|vcpus" -From ([DateTime]::Now).AddMinutes(-120) | Select-Object -Last 1).Value
}
else {
$DownSizeMem = 0
$DownSizeCPU = 0
}
# Mem is in KB
if($UpSize -gt 0){
$UpSizeMem = ($OMResource | Get-OMStat -Key "summary|undersized|memory" -From ([DateTime]::Now).AddMinutes(-120) | Select-Object -Last 1).Value
$UpSizeCPU = ($OMResource | Get-OMStat -Key "summary|undersized|vcpus" -From ([DateTime]::Now).AddMinutes(-120) | Select-Object -Last 1).Value
}
else {
$UpSizeMem = 0
$UpSizeCPU = 0
}
$Report = [PSCustomObject] @{
Name = $OMResource.name
DownSize = $DownSize
UpSize = $UpSize
DownSizeMem = $DownSizeMem
DownSizeMemGB = [Math]::Round(($DownSizeMem / 1048576), 0)
DownSizeCPU = $DownSizeCPU
UpSizeMem = $UpSizeMem
UpSizeMemGB = [Math]::Round(($UpSizeMem / 1048576), 0)
UpSizeCPU = $upSizeCPU
}
$View += $Report
}
$View | Sort-Object DownSizeMemGB, DownSizeCPU -Descending | Format-Table -AutoSize
You can play a little with the generated list by piping it to the great Get-GridView Cmdlet:
Now that we have a list of all Sizing Recommendations for the given list of VMs, there are just a few extra lines of code necessary to apply the recommendations.
- Shutdown Guest OS
- Take Snapshot
- Apply Rightsizing Recommendations (Up- and Downsizing or only Downsizing)
- [Update CPU and Memory reservation]
- PowerOn VM
Get-Module -Name VMware* -ListAvailable | Import-Module -Force Connect-VIServer vCenter01 Connect-OMServer -AuthSource "AD" -Server vRops01 Get-VM -Name test-* | Get-OMResource | Apply-OMRightsizing -ViewOnly | Sort-Object DownSizeMemGB, DownSizeCPU -Descending | Format-Table -AutoSize Get-VM -Name test-* | Get-OMResource | Apply-OMRightsizing -Apply -NoUpsizing
function Apply-OMRightsizing {
<#
.NOTES
===========================================================================
Created by: Markus Kraus
===========================================================================
Changelog:
2020.07 ver 1.0 Base Release
===========================================================================
External Code Sources:
-
===========================================================================
Tested Against Environment:
vSphere Version: vSphere 6.7 U3
PowerCLI Version: PowerCLI 11.5
PowerShell Version: 5.1
OS Version: Windows 10
Keyword: vSphere, vRealize, Rightsizing
===========================================================================
.DESCRIPTION
This function views or applies rightsizing recommendations from vRealize Operations to your vSphere VMs.
.Example
Get-VM -Name test-* | Get-OMResource | Apply-OMRightsizing -ViewOnly | Sort-Object DownSizeMemGB, DownSizeCPU -Descending | Format-Table -AutoSize
.Example
Get-VM -Name test-* | Get-OMResource | Apply-OMRightsizing -Apply -NoUpsizing
.PARAMETER OMResources
vRealize Operations Ressources to process
.PARAMETER ViewOnly
View Recommendations
.PARAMETER Apply
Apply Recommendations
.PARAMETER NoUpsizing
Apply only Downsizing Recommendations
#Requires PS -Version 5.1
#Requires -Modules VMware.VimAutomation.Core, @{ModuleName="VMware.VimAutomation.Core";ModuleVersion="11.5.0.0"}
#>
[CmdletBinding()]
param(
[Parameter(Mandatory=$True, ValueFromPipeline=$True, Position=0, HelpMessage = "OM Ressources to process")]
[ValidateNotNullorEmpty()]
$OMResources,
[Parameter(Mandatory=$False, ValueFromPipeline=$False, ParameterSetName="ViewOnly", HelpMessage = "View Recommendations")]
[Switch] $ViewOnly,
[Parameter(Mandatory=$False, ValueFromPipeline=$False, ParameterSetName="Apply", HelpMessage = "Apply Recommendations")]
[Switch] $Apply,
[Parameter(Mandatory=$False, ValueFromPipeline=$False, ParameterSetName="Apply", HelpMessage = "Apply only Downsizing Recommendations")]
[Switch] $NoUpsizing
)
Process {
if ($ViewOnly -or $Apply){
"Collecting Report ..."
$View = @()
foreach ($OMResource in $OMResources){
$DownSize = ($OMResource | Get-OMStat -Key "summary|oversized" -From ([DateTime]::Now).AddMinutes(-120) | Select-Object -Last 1).Value
$UpSize = ($OMResource | Get-OMStat -Key "summary|undersized" -From ([DateTime]::Now).AddMinutes(-120) | Select-Object -Last 1).Value
# Mem is in KB
if($DownSize -gt 0){
$DownSizeMem = ($OMResource | Get-OMStat -Key "summary|oversized|memory" -From ([DateTime]::Now).AddMinutes(-120) | Select-Object -Last 1).Value
$DownSizeCPU = ($OMResource | Get-OMStat -Key "summary|oversized|vcpus" -From ([DateTime]::Now).AddMinutes(-120) | Select-Object -Last 1).Value
}
else {
$DownSizeMem = 0
$DownSizeCPU = 0
}
# Mem is in KB
if($UpSize -gt 0){
$UpSizeMem = ($OMResource | Get-OMStat -Key "summary|undersized|memory" -From ([DateTime]::Now).AddMinutes(-120) | Select-Object -Last 1).Value
$UpSizeCPU = ($OMResource | Get-OMStat -Key "summary|undersized|vcpus" -From ([DateTime]::Now).AddMinutes(-120) | Select-Object -Last 1).Value
}
else {
$UpSizeMem = 0
$UpSizeCPU = 0
}
$Report = [PSCustomObject] @{
Name = $OMResource.name
DownSize = $DownSize
UpSize = $UpSize
DownSizeMem = $DownSizeMem
DownSizeMemGB = [Math]::Round(($DownSizeMem / 1048576), 0)
DownSizeCPU = $DownSizeCPU
UpSizeMem = $UpSizeMem
UpSizeMemGB = [Math]::Round(($UpSizeMem / 1048576), 0)
UpSizeCPU = $upSizeCPU
}
$View += $Report
}
}
if ($ViewOnly){
$View
}
if ($Apply){
foreach ($Object in $View) {
if ($Object.DownSize -gt 0 -or $Object.UpSize -gt 0){
"Processing '$($Object.Name)' ..."
$VM = Get-VM -Name $Object.Name
"Shut down '$($Object.Name)' ..."
$VM | Shutdown-VMGuest -Confirm:$False
$i = 0
while((Get-VM -Name $VM.Name).PowerState -eq "PoweredOn"){
$i++
Start-Sleep 1
Write-Progress -Activity "Check PowerState" -Status "Wait for PowerState Task..."
}
"Create Snapshot for '$($Object.Name)' ..."
$VM | New-Snapshot -Name "Pre Resize" -Memory:$false -Quiesce:$false
if ($Object.DownSize -gt 0){
"Downsize '$($Object.Name)' ..."
$VM | Set-VM -NumCPU $($VM.NumCpu - $Object.DownSizeCPU) -MemoryGB $($VM.MemoryGB - $Object.DownSizeMemGB) -Confirm:$False
}
if ($Object.UpSize -gt 0 -and $NoUpsizing -eq $False){
"Upsize '$($Object.Name)' ..."
$VM = Get-VM -Name $Object.Name
$VM | Set-VM -NumCPU $($VM.NumCpu + $Object.UpSizeCPU) -MemoryGB $($VM.MemoryGB + $Object.UpSizeMemGB) -Confirm:$False
}
#$VM = Get-VM -Name $Object.Name
#$VM | Get-VMResourceConfiguration | Set-VMResourceConfiguration -CpuReservationMhz $($VM.NumCpu * 200) -MemReservationGB $($VM.MemoryGB / 2) -Confirm:$False
"Power on '$($Object.Name)' ..."
$VM | Start-VM -Confirm:$False
}
}
}
}
}
Warning:
This script is designed for a specific use case and is not tested against all possible constellations.
The Pull request to the VMware PowerCLI Examples Repository that includes this function is currently in review.
Additional References
- VM Rightsizing Details Dashboard for vRealize Operations 8.0 8.1 and Cloud
- Rightsizing VMs with vRealize Operations
- Enhancements to Virtual Machine Memory Metrics in vRealize Operations
- ESXi Cluster (non-HCI) Rightsizing using vRealize Operations
No Responses