- HP servers running ESXi 7.0.
- VMware Datastores are on Netapp Ontap Select storage software, which is pooling together locally attached Intel S3710 SSDs, and presenting this clustered pool of locally attached storage over iSCSI.
- Customer wanted very low VM latencies for their write intensive ESRI GIS Mapping applications that are running within Nvidia vGPU assisted Horizon View VDI VMs.
- VirtuCache is installed in ESXi 7 in each host, and it is configured to cache to in-host 1.5TB Intel P4800x Optane NVME SSD. In VirtuCache, 'Write-Back One Replica' policy is applied to all the Datastores. 'Write-Back One Replica' policy caches reads and writes to in-host cache media and replicates writes to cache media in another host in the cluster. The write cache replication happens over the customer's 40g vMotion network.
- VirtuCache ensures that storage latencies are consistently under 5ms at the VM level for the customer's ESRI GIS Mapping application.
The Virtunet DifferenceThe customer selected VirtuCache because:
April 15, 2020:
City of Mountain View is using VirtuCache to improve the performance of Netapp’s Ontap Select storage software. They were one of our first customers to upgrade to ESXi 7.0 and so they had to upgrade VirtuCache as well.
We announced General Availability of VirtuCache for ESXi 7 within 2 weeks of VMware’s ESXi 7 release date.
Storage performance problems in Netapp Ontap Select
The customer’s storage is running on Ontap Select storage software that in turn is using locally attached Intel S3710 SATA SSDs. Ontap Select is software from Netapp that you install in VMware VMs, it then pools locally attached storage media and presents it to the VMware hosts over iSCSI or NFS. The main drawback with Ontap Select is that it is very slow even when using enterprise grade low latency SSDs as is the case here. We think this is because there are many file system layers that the storage IO path has to traverse. With Ontap Select, you first create a separate VMFS Datastore on each raw SSD or HDD that’s attached to an ESXi host. Thereafter using Ontap Select you cluster these individual Datastores, deploy Netapp’s file system on this clustered pool of locally attached storage, and present it over iSCSI or NFS. Then the VMware admin creates regular VMFS Datastores for VMs. So the storage IO path from the VM goes through VMFS >> Netapp File System >> VMFS again (since a VMFS Datastore has to be initially created on each raw locally attached HDD / SSD) >> raw block device SSD / HDD attached to the ESXi host. We think that such a long winded storage IO path is the reason for high latencies in Ontap Select.
VirtuCache was deployed with Intel Optane P4800X PCIe SSDs
In comparison with regular enterprise NVME SSDs that are high performing for reads but not as much for writes, Intel’s Optane SSDs are equally high performing for both writes and reads. Optane SSDs are rated at 500K Random Read / Write IOPS. Also, they can sustain 160 PetaBytes of total lifetime writes, so they also last a very long time. Their high endurance and write performance is more in line with DRAM than NAND Flash. Since the customer’s workload is mission critical, write intensive, and high throughput, we decided to use Optane SSDs with VirtuCache.
Cost to Customer
This solution costed the customer US $ 9600 per host [VirtuCache cost of $3600/host + $6000 for the 1.5TB Intel P4800X Optane SSD].
Reason for upgrading to ESXi 7
The customer is running GIS applications within their VDI VMs. Virtual GPUs (vGPUs) are assigned to these VDI VMs, and the GIS Mapping applications rely heavily on vGPUs. Hence it was important that these vGPU assisted VDI VMs support vSphere enterprise features like HA and vMotion. ESXi 7 has a new feature called ‘Assignable Hardware’, that makes it easier for VMs with attached vGPUs to vMotion and restart on other hosts in case of a host failure, and so the customer decided to upgrade to ESXi 7.
Signup for the VirtunetSystems Newsletter