CEPH is a popular open source storage software. However its write latencies are high. VirtuCache caches CEPH volumes to in-host SSDs and by doing so reduces VM level latencies considerably.
Fairbanks Sewer and Water (FSW) wanted to improve the performance of their server VMs (especially their server VMs running SCADA) and Horizon View virtual desktop VMs. Their virtual desktop VMs were exhibiting all the classic symptoms of storage related issues – cursor freezes, long boot times, long VM provisioning times, jittery audio/video.
VirtuCache is software that improves the performance of Equallogic appliances without requiring you to upgrade the appliance or the SAN network. The performance improvement you will get from your Equallogic appliance will rival an upgrade to an all-flash array.
Equallogic appliances were the workhorses of the enterprise storage market a few years ago. They were cost effective at high capacities. The only drawback was that they are slower since they are primarily hard drive based, and when connected to VMware they exhibit all the 'IO blender' symptoms resulting in high VM level storage latencies.
ServiceNow's Itapp Dev/Ops team wanted to improve storage performance from their existing HP 3PAR storage appliance and iSCSI storage network without requiring a hardware refresh.
VirtuCache Deployment: Virtucache was installed on 3 ESXi hosts caching to 1.6TB PM1725 PCIE flash cards. In our tests the PM1725 SSD did 250MBps at 1ms VM level latencies.
VirtuCache was configured to cache both reads and writes for all VMs (Write-Back caching). Writes were replicated to another SSD on another host (caching policy of 'Write Back One Replica'). All caching and replication related operations in VirtuCache are automatic. Write replication is done to prevent data loss in case of host failure. If a host were to fail, then VirtuCache immediately syncs the SAN from backup copy of writes that are on another host.
Benefits: Using VirtuCache, ServiceNow was successfully able to reduce code compile times to a third of what they were experiencing before.
By not using dedupe, compression, or RAID, using slow HDDs in centralized storage, and moving SSDs to compute hosts, we arrived at low price per capacity and performance for video storage.
Here are unique requirements of video storage, some are obvious and others not so much, that inspired us to put together a different architecture than the conventional storage OEM design.
VirtuCache is installed in VMware vSphere kernel. It then automatically caches frequently and recently used data from any backend storage to any high speed media (RAM/SSD) in the VMware host. By bringing large amounts of 'hot' data closer to the VMware host GPU and CPU, VirtuCache improves the performance of all applications running within VMs including GPU assisted operations.
There are only a few applications, financial trading software being one example, that require very low latencies, lower even than what’s possible with an all-flash array (AFA). VirtuCache caching to in-host RAM results in lower VM latencies than an AFA. This is because RAM latencies are an order of magnitude lower than NVME SSDs, and in the case of VirtuCache the cache media (RAM) is connected to the host CPU through a high speed memory bus, versus in the case of an AFA where the NVME SSDs are behind the network and storage controller.
High write latencies in a stretched SAN cluster
Tourbillon Capital Partner is a hedge fund. They run proprietary trading software within VMware VMs that requires under 5 millisecond latencies. Tourbillon has two VMware clusters with a few nodes in each cluster. Each ESXi cluster is connected to a Pure Storage SAN array. Both ESXi clusters are in different datacenters, but connected to each other over a 10gbps WAN link. A stretched SAN cluster across these two ESXi clusters is created using Datacore software. Simply speaking what the Datacore stretched cluster accomplishes is that all VM writes are synchronously written to both Pure Storage arrays - the array that’s in the same datacenter as the VM, and also to the remote Pure Storage array. In this way Tourbillon’s IT folks assure themselves of seconds-to-minutes RPO and RTO time in case of a datacenter outage.
The problem with this architecture was that sometimes VM write latencies exceeded the 10ms ceiling that was required by their trading application. This was because writes had to go over their WAN link between datacenters. Even though the WAN link was 10gbps, it would spike to > 5ms latencies from time to time. Typically, the standard deviation for latencies in a long distance WAN link is quite a bit more than in shorter LAN links of the same speed.
Caching VM writes to in-host RAM reduced write latencies considerably
Tourbillon deployed VirtuCache to fix this issue. VirtuCache was installed in every host, in both ESXi clusters. It was configured to cache reads and writes to in-host RAM, with the write cache replicated to another host in the same datacenter, which in turn resulted in sub-millisecond VM write latencies at all times. In this way, VirtuCache effectively papered over the underlying high WAN latencies, when large volume of writes were transmitted from VMs.
Dell's PowerEdge VRTX hyper-converged appliance can either have all hard drive datastores or all SSD datastores, but you can't have SSDs act as tiering or caching media for VRTX volumes. That's where VirtuCache comes in.
Creation Museum in Kentucky, USA is a museum about Bible history and creationism. Their storage needs were typical of a museum, requiring large amounts of storage for digital multimedia content related to the various exhibits at the museum.
The Ark Encounter, in Williamstown, Kentucky, features a full-size Noah’s Ark built according to the dimensions of the Bible. Answers in Genesis (AiG) is the Christian ministry responsible for The Ark Encounter.
AiG's IT department had a few ESXi hosts connected to their HP Store VSA. As a result of increased attendance at the Ark, their VMware workload increased dramatically, which in turn resulted in performance issues within VMs.
AiG turned to VirtuCache to mitigate their storage latency issues. By caching frequently and recently used data (both reads and writes) to in-host SSDs+RAM, Virtunet resolved their storage performance issues. We competed with HP Store VSA's Adaptive Optimization(AO) feature, which is HP's tiering functionality for the VSA.
Here is how VirtuCache competes with the Store VSA's tiering functionality.