Setting IOPS limit for path switch over for Round Robin PSP
Some storage vendors recommend to use Round Robin PSP and to switch between paths for every IOP. By default, for the VMware Round Robin PSP, storage IO switches between paths when IOPS count of 1000 is reached, but some storage vendors recommend switching paths for every IOP (IOPS count of 1). VMware supports this feature in its Round Robin PSP.
We support it in the VirtuCache Round Robin PSP as well (VNX_PSP_RR). Just like with the VMware RR PSP, our default IOPS count for switching paths is 1000, but you can adjust it to any IOPS count.
Below is the esxcli command to set the IOPS count to 1 for VirtuCache Round Robin PSP (VNX_PSP_RR).
Run the below command after VirtuCache is installed.
esxcli storage nmp psp generic deviceconfig set -d -c ‘type=iops,iops=1’
esxcli storage nmp psp generic deviceconfig set -d naa.60000000000000000e00000000010001 -c ‘type=iops,iops=1’
NB: This setting persists on an ESXi host reboot starting in VirtuCache build 2728 for ESXi 6.x and 2814 for ESXi 7.x
If you want to change the IOPS setting across a large number of paths, then you could use the attached python script. change_rr_iops. Unzip the file and transfer it to a datastore (or /tmp/ folder in each ESX host), and then run this command from ESXi CLI:
python /tmp/change_rr_iops.py –iops=1
This script first checks to see if all paths have the Virtunet RR PSP (VNX_PSP_RR), if some paths still have VMware’s PSP (VMW_PSP_RR), then it will replace VMW_PSP_RR with VNX_PSP_RR and then change the IOPS count of the VirtuCache RR PSP to 1. If the LUNs don’t already have VMware’s VMW_PSP_RR assigned to it, then you would need to do that first. Please consult with your storage array vendor regarding the PSP to be used with their LUNs, this recommendation comes solely from the array vendor.
Note that even though VMware doesn’t require that the host be put in maintenance mode for changing the PSP or IOPS value, the throughput/IOPS will drop when the PSP is changed, since switching PSP involves SCSI Unclaim and re-Claim operations. During Unclaim operation throughput/IOPS is affected and if VirtuCache is running in Write-Back caching mode, Unclaim might take upto 2 minutes because the VirtuCache write cache needs to be flushed to SAN array, so we recommend running the script in maintenance mode.