How to find out the Adapter Queue Depth and Device Queue Depth of RAID controller and RAIDed SSD?
Importance of Queue Depth – Higher the Queue Depth, Lower the Latency
Queue Depth is a number assigned by the device vendor to their storage device that advertises to the component above the device in the storage IO path, the maximum number of IO requests the device can process simultaneously. Every device or software component in the storage IO path has a Queue Depth associated with it. IO requests that are sent to that device, that are in excess of the Queue Depth, get queued. There should be minimal queuing on any device in the storage IO path, else the latencies will increase a lot.
Queue Depths for SATA / SAS SSDs
If using SATA or SAS SSDs, check the Queue Depth of the SSD device and the RAID controller. In the case of cheap (low Queue Depth) RAID controllers, the RAID controller becomes a bottleneck to the RAID0 SSD behind it, hence it is very important that the Queue Depth for the RAID0 SSD device (Device Queue Depth or DQLEN field in ESXi) and the Queue Depth for the RAID controller (Adapter Queue Depth or AQLEN field in ESXi) be greater than 512.
Queue Depths for NVME SSDs
In the case of NVME SSDs, Queue Depths (AQLEN and DQLEN) are almost always very high (greater than 900). But it’s a good idea to confirm this.
Using ESXTOP to find out values of AQLEN and DQLEN
To find out the Queue Depth (AQLEN) for the adapter e.g. RAID controller or NVME Adapter, do this – SSH to the host using putty, type esxtop at the ESXi CLI, then type d, then type f (to show additional fields), then type d (to turn on ‘Queue Stats’), hit enter (to get back to prior screen), and note down the AQLEN field for the adapter.
To find out Device Queue Depth (DQLEN) for the device e.g. SSD(s) that are in RAID0 configuration by the RAID controller, or the NVME SSD, do this – type esxtop at the ESXi CLI, then type u, and note down the DQLEN field for the device.
Both AQLEN and DQLEN fields should be greater than 512 to be assured that the SSD will perform well.