Scroll Top

Reducing write latencies in a VMware stretched SAN cluster

Reducing write latencies in a VMware stretched SAN cluster

Use Case:
Storage Performance

Location:
London

Challenges:

  • Customer had deployed a stretched SAN cluster that spanned datacenters. This was done for seconds-to-minutes RPO and RTO time in case an entire datacenter failed. Because of the stretched cluster, writes were synchronously committed across the WAN link between datacenters. This resulted in high write latencies. Because the customer was in the high frequency trading business, they wanted sub 5 millisecond write latencies for their trading software. So their main problem was to achieve 5ms VM write latencies despite the inconsistent latencies over the WAN link.

Benefits:

  • Virtucache ensured that write latencies were consistently under 5ms at the VM level in the customer’s stretched SAN ESXi cluster that was spanned a WAN link.

Reducing write latencies in a VMware stretched SAN cluster

The Virtunet Difference
  • The customer went with us because VirtuCache is the only host side caching software that could do these 3 things : accelerate writes; cache to host RAM; restrict write cache replica to hosts in just the local datacenter in a stretched SAN cluster.
  • The only other solution was to upgrade the WAN link which was very expensive.

    • There are only a few applications, financial trading software being one example, that require very low latencies, lower even than what’s possible with an all-flash array (AFA). VirtuCache caching to in-host RAM results in lower VM latencies than an AFA. This is because RAM latencies are an order of magnitude lower than NVME SSDs, and in the case of VirtuCache the cache media (RAM) is connected to the host CPU through a high speed memory bus, versus in the case of an AFA where the NVME SSDs are behind the network and storage controller.

    High write latencies in a stretched SAN cluster

    Tourbillon Capital Partner is a hedge fund. They run proprietary trading software within VMware VMs that requires under 5 millisecond latencies. Tourbillon has two VMware clusters with a few nodes in each cluster. Each ESXi cluster is connected to a Pure Storage SAN array. Both ESXi clusters are in different datacenters, but connected to each other over a 10gbps WAN link. A stretched SAN cluster across these two ESXi clusters is created using Datacore software. Simply speaking what the Datacore stretched cluster accomplishes is that all VM writes are synchronously written to both Pure Storage arrays – the array that’s in the same datacenter as the VM, and also to the remote Pure Storage array. In this way Tourbillon’s IT folks assure themselves of seconds-to-minutes RPO and RTO time in case of a datacenter outage.

    The problem with this architecture was that sometimes VM write latencies exceeded the 10ms ceiling that was required by their trading application. This was because writes had to go over their WAN link between datacenters. Even though the WAN link was 10gbps, it would spike to > 5ms latencies from time to time. Typically, the standard deviation for latencies in a long distance WAN link is quite a bit more than in shorter LAN links of the same speed.

    Caching VM writes to in-host RAM reduced write latencies considerably

    Tourbillon deployed VirtuCache to fix this issue. VirtuCache was installed in every host, in both ESXi clusters. It was configured to cache reads and writes to in-host RAM, with the write cache replicated to another host in the same datacenter, which in turn resulted in sub-millisecond VM write latencies at all times. In this way, VirtuCache effectively papered over the underlying high WAN latencies, when large volume of writes were transmitted from VMs.

    VirtuCache reducing write latencies in a stretched SAN cluster

    Download Trial Contact Us