Installation and Configuration Instructions for VirtuCache in a VMware Stretched Cluster

Terms to Note.

‘Write-through’ caching policy in VirtuCache is where only reads are accelerated.

‘Write-back One Replica’ is a caching policy where both reads and writes are accelerated. And the writes are replicated to cache media in another host in the same ESXi cluster.

Step1: Deploy VirtuCache OVA.

1. Import the VirtuCache Manager OVA to a vCenter that has IP connectivity to the ESXi hosts where VirtuCache caching driver software will be installed. If your network has static IPs, assign the VM an IP address during the import process. VirtuCache Manager is our Graphical User Interface.¹ It is not in the storage IO path.

Step 2: Select In-VMware Host Caching Media.

2. You want 10-20% of storage used by all VMs on the host to be cache media capacity in the host. This will ensure that more than 90% of the requests are serviced from cache (cache hit %).

3. If you have spare host RAM, then please use that, since RAM is the highest performing cache media. But please also make sure you have enough spare RAM available to make a difference (point #2 above).

4. If you don’t have adequate spare RAM on the host, but if you have a spare PCIe or U.2 slot, then we highly recommend that you use an enterprise-grade PCIe / NVME SSD (costs only 30 US cents/GB). A single 1.6TB NVME SSD in PCIe form factor per host is the most popular cache media at our customers.

5. If you don’t have enough spare RAM, a PCIe slot, or a U.2 slot then enterprise SATA/SAS SSD is the only option. Please don’t use consumer SSDs, since their latencies are high. If using SATA/SAS SSD, preferably use only one SSD per host and RAID0 the SSD. RAID1 deteriorates SSD performance and is not required since VirtuCache takes care of data protection. Also, make sure that you have a high queue depth RAID controller, else the RAID controller will be the bottleneck to the SATA/SAS SSD. More on RAID controller and SSD selection here.

Step 3: Install VirtuCache Software and License on ESXi Host(s).

6. VirtuCache driver needs to be installed in every host in the cluster.

7. Install VirtuCache driver software vib of the type “vnx2-1.0-.*.vib” using ESXi CLI like below.

   esxcli software vib install -v "file:///vmfs/volumes/[datastore name]/vnx2-1.0-.*.vib"

   where [datastore name] is the Datastore the vib has been copied to.

8. Get license key file of the type *lic_vnx2*.vib from Virtunet and install it in the host much like the VirtuCache driver, using the same ‘esxcli software vib install’ command listed in #7 above.

9. This step is two tasks rolled into one. (a) Assigning cache device to ESXi hosts and (b) Ensuring that the write latency is low by disallowing the mirrored copy of write cache from traversing the inter-site WAN link. In the case of Stretched Cluster, both steps 9(a) and 9(b) listed here need to be performed from VirtuCache CLI and not from the GUI (as is normally the case in a regular VMware cluster). Follow this Knowledgebase article to perform this step.

Step 4: Configure Caching on ESXi Host(s).

10. Using Chrome or Firefox, connect to VirtuCache Manager IP address. Login using vCenter credentials. The GUI might take up to 5 minutes to populate all the information on the various tabs when invoked for the first time.

11. On the ‘Hosts’ tab, select ‘Replication Network’. This network is used for mirroring cached writes to another host and for the read cache to follow the VM in case of vmotion. By default, we use the vMotion network as the replication network, though you can use a different network. Ensure that the VMkernel adapter associated with the ‘Replication Network’ is configured with VMware’s ‘default’ TCP/IP stack, and not the ‘vMotion’ stack, else VMware will not let VirtuCache traffic pass. For low latency writes, a 10g network with Jumbo Frames is highly recommended. Lastly, there should be connectivity between the Replication Networks at both sites, else you will have a problem with inter-site vmotion.

12. On the ‘Hosts’ tab, under the column called ‘Cache’, you should see the SSD / RAM assigned to VirtuCache from step 9 above. Please be aware that VirtuCache takes over the entire SSD and RAM assigned to it.

13. Goto the ‘Datastores’ or ‘Virtual Machines’ tab and configure Datastores and/or VMs with a caching policy. Configure ‘Caching Policy’ ^2,3 as ‘Write-back One Replica’ if you need to accelerate both reads and writes, or as ‘Write-through’ if only reads need to be accelerated. You can assign a different caching policy to the VM(s) than what it inherits from the Datastore by going to the ‘Virtual Machines’ tab.

14. Goto the’ Hosts’ tab and ensure that caching is working on all hosts. Datastores / VMs configured with a caching policy will now be accelerated.

Step 5: Measure Performance.

15. View stats at the Host and Datastore level by clicking on a Host on the ‘Hosts’ tab. View VM level performance stats by going to the ‘Performance’ tab. If VM level read and write latencies are under 5ms, then VirtuCache and the cache media are performing as expected.

Things to watch out for.

The most common reasons for VirtuCache not improving performance are high SSD latency (# 5 above); when using only RAM such small amounts of RAM are assigned to VirtuCache that the cache gets full quickly but the cache hit % remains low (#2 above); and high network latency (# 11 above).