CEPH Storage with Server Side Caching for Genomics
March 7, 2019:
As it relates to storage, genome sequencing workflow has two requirements.
-
Need for cheap ($/GB) Petabyte scale storage.
-
Need for faster processing of Genome files under analysis, many of which are a few hundred GB in size.
Reducing Build Times for DevOps
April 12, 2018:
ServiceNow's Itapp Dev/Ops team wanted to improve storage performance from their existing HP 3PAR storage appliance and iSCSI storage network without requiring a hardware refresh.
VirtuCache Deployment: Virtucache was installed on 3 ESXi hosts caching to 1.6TB PM1725 PCIE flash cards. In our tests the PM1725 SSD did 250MBps at 1ms VM level latencies.
VirtuCache was configured to cache both reads and writes for all VMs (Write-Back caching). Writes were replicated to another SSD on another host (caching policy of 'Write Back One Replica'). All caching and replication related operations in VirtuCache are automatic. Write replication is done to prevent data loss in case of host failure. If a host were to fail, then VirtuCache immediately syncs the SAN from backup copy of writes that are on another host.
Benefits: Using VirtuCache, ServiceNow was successfully able to reduce code compile times to a third of what they were experiencing before.
A Counter Intuitive Approach to Solve the High Capacity, Low Latency Requirements for Video Storage
June 14, 2017:
By not using dedupe, compression, or RAID, using slow HDDs in centralized storage, and moving SSDs to compute hosts, we arrived at low price per capacity and performance for video storage.
Here are unique requirements of video storage, some are obvious and others not so much, that inspired us to put together a different architecture than the conventional storage OEM design.
Reducing latencies in vGPU assisted VDI
October 1, 2016:
VirtuCache is installed in VMware vSphere kernel. It then automatically caches frequently and recently used data from any backend storage to any high speed media (RAM/SSD) in the VMware host. By bringing large amounts of 'hot' data closer to the VMware host GPU and CPU, VirtuCache improves the performance of all applications running within VMs including GPU assisted operations.
VirtuCache on ESXi 7, accelerating Netapp Ontap Select Storage
April 15, 2020:
City of Mountain View is using VirtuCache to improve the performance of Netapp's Ontap Select storage software. They were one of our first customers to upgrade to ESXi 7.0 and so they had to upgrade VirtuCache as well.
We announced General Availability of VirtuCache for ESXi 7 within 2 weeks of VMware's ESXi 7 release date.
Reducing write latencies in a VMware stretched SAN cluster
October 12, 2018:
There are only a few applications, financial trading software being one example, that require very low latencies, lower even than what’s possible with an all-flash array (AFA). VirtuCache caching to in-host RAM results in lower VM latencies than an AFA. This is because RAM latencies are an order of magnitude lower than NVME SSDs, and in the case of VirtuCache the cache media (RAM) is connected to the host CPU through a high speed memory bus, versus in the case of an AFA where the NVME SSDs are behind the network and storage controller.
High write latencies in a stretched SAN cluster
Tourbillon Capital Partner is a hedge fund. They run proprietary trading software within VMware VMs that requires under 5 millisecond latencies. Tourbillon has two VMware clusters with a few nodes in each cluster. Each ESXi cluster is connected to a Pure Storage SAN array. Both ESXi clusters are in different datacenters, but connected to each other over a 10gbps WAN link. A stretched SAN cluster across these two ESXi clusters is created using Datacore software. Simply speaking what the Datacore stretched cluster accomplishes is that all VM writes are synchronously written to both Pure Storage arrays - the array that’s in the same datacenter as the VM, and also to the remote Pure Storage array. In this way Tourbillon’s IT folks assure themselves of seconds-to-minutes RPO and RTO time in case of a datacenter outage.
The problem with this architecture was that sometimes VM write latencies exceeded the 10ms ceiling that was required by their trading application. This was because writes had to go over their WAN link between datacenters. Even though the WAN link was 10gbps, it would spike to > 5ms latencies from time to time. Typically, the standard deviation for latencies in a long distance WAN link is quite a bit more than in shorter LAN links of the same speed.
Caching VM writes to in-host RAM reduced write latencies considerably
Tourbillon deployed VirtuCache to fix this issue. VirtuCache was installed in every host, in both ESXi clusters. It was configured to cache reads and writes to in-host RAM, with the write cache replicated to another host in the same datacenter, which in turn resulted in sub-millisecond VM write latencies at all times. In this way, VirtuCache effectively papered over the underlying high WAN latencies, when large volume of writes were transmitted from VMs.
VMware SSD and RAM Cache for Dell VRTX
October 11, 2018:
Dell's PowerEdge VRTX hyper-converged appliance can either have all hard drive datastores or all SSD datastores, but you can't have SSDs act as tiering or caching media for VRTX volumes. That's where VirtuCache comes in.
PetaByte Scale Enterprise Grade Server SAN Storage for The Creation Museum
September 17, 2017:
Creation Museum in Kentucky, USA is a museum about Bible history and creationism. Their storage needs were typical of a museum, requiring large amounts of storage for digital multimedia content related to the various exhibits at the museum.
Improving Storage Performance at The Ark
April 28, 2018:
The Ark Encounter, in Williamstown, Kentucky, features a full-size Noah’s Ark built according to the dimensions of the Bible. Answers in Genesis (AiG) is the Christian ministry responsible for The Ark Encounter.
AiG's IT department had a few ESXi hosts connected to their HP Store VSA. As a result of increased attendance at the Ark, their VMware workload increased dramatically, which in turn resulted in performance issues within VMs.
AiG turned to VirtuCache to mitigate their storage latency issues. By caching frequently and recently used data (both reads and writes) to in-host SSDs+RAM, Virtunet resolved their storage performance issues. We competed with HP Store VSA's Adaptive Optimization(AO) feature, which is HP's tiering functionality for the VSA.
Here is how VirtuCache competes with the Store VSA's tiering functionality.
CEPH Storage is Ideally Suited for Disaster Recovery (DR) Infrastructure
July 12, 2017:
CEPH storage from Virtunet has all the features of traditional iSCSI SAN with the exception that it is reasonably priced because it uses commodity servers with all off-the-shelf hardware. And so it is ideally suited for backup and DR storage which needs to be cheap above all else.