Archive: Posts

The reason for high latency in Ontap Select and how to fix it?

Even when Ontap Select is using high-performance enterprise SSDs, it shows high latencies because there are quite a few file system layers that the storage IO must traverse. Within Ontap Select, you first create a separate VMFS Datastore on each locally attached SSD or HDD on each host in the ESXi cluster; thereafter you pool together these individual VMware Datastores across hosts; deploy Netapp’s file system on this clustered pool of VMware Datastores; then present it over iSCSI or NFS. Also, all data is replicated to media in another ESXi host over a VMware network. Finally, on top of this clustered pool of Datastores, you again create VMFS Datastores, this time for VM storage. So, the storage IO path from the VM goes through VMFS >> Netapp File System >> VMware network >> VMFS again >> in-host SSD / HDD. Such a long-winded storage IO path is the reason for high latencies in Ontap Select.

VirtuCache configuration to improve Ontap Select performance?

VirtuCache works with Ontap Select storage in the same way that it works with traditional SAN storage appliances. All reads and writes from VMs that are on Ontap Select storage are cached by VirtuCache to in-ESXi host SSD or RAM. The only caveat is that Ontap Select needs to be configured over iSCSI and not NFS for VirtuCache to work.

In VirtuCache, you would apply the ‘Write-Back 1 Replica’ caching policy to Datastores or VMs. This policy caches all reads and writes from VMs to the in-host RAM / SSD that you assigned to VirtuCache. It also mirrors the write cache to cache media in another host in the ESXi cluster. This is to protect against data loss if a host were to fail. For more details on how VirtuCache does this, please review the ‘Cache Replication’ section on this link.

With VirtuCache in the storage IO path, cached read requests go from VM to VirtuCache driver software >> in-host SSD / RAM and the writes go from VM >> VirtuCache driver software >> VMware network (used for mirroring writes) >> in-host SSD / RAM. As a result of the shorter IO path, VM storage latencies are much reduced.

Before / After storage IO performance testing using Iometer.

Below are results from an Iometer test performed from within a VM that resides on Ontap Select Datastore, with and without VirtuCache. It is a straightforward 75/25 Random Read/Random Write test using 4KB block size. Ontap Select was running on locally attached four 960GB Samsung PM863 Enterprise SATA SSDs in RAID5 config in each host in the cluster, and it was mirroring all data between hosts over a 40gb network.

VirtuCache configuration.

VM Read MBps

VM Write MBps

VM Read Latency

VM Write Latency

No VirtuCache.





VirtuCache in ‘Write-Back 1 Replica mode’, caching to RAM.






More valuable than the stats shown in the table above, is the fact that for real-life customer workload, VirtuCache ensures that VM level read and write latencies are always under 5ms, and thus VirtuCache allows for the deployment of latency sensitive workloads on Ontap Select.