Write Back caching policy

Write-Back caching policy accelerates reads and writes.

The read IO path is exactly the same as with Write-Through caching. However, the write IO path is very different. All writes from VMware are written to the in-host cache media without synchronously writing to the backend storage. By writing only to the in-host cache media, writes are substantially accelerated, The fact that VirtuCache is not synchronously committing the writes to the backend storage appliance introduces the risk of data loss in case the local host were to fail. To guard against this possibility, VirtuCache protects the local write cache by replicating (mirroring) the writes across hosts in a VMware cluster.

Write cache replication to protect against data loss in case of host failure.

VirtuCache clusters together cache media across hosts in an ESXi cluster. The main reason we do this is to be able to mirror the write cache across VMware hosts in a distributed fashion. The administrator specifies the number (0, 1, or 2) of copies of write cache that need to be on separate hosts for each local VM write cache in the cluster. We call this a ‘write replica’. The number of write replicas indicates the maximum number of node failures that can be sustained before there is data loss in the cluster. If an administrator chooses to keep, say, one write replica (we call this ‘Write-Back 1 Replica’ policy), VirtuCache automatically replicates the dirty writes (dirty writes are writes in the host cache that have not yet been synced with backend storage) to cache media on one additional VMware host in the same cluster. VirtuCache defaults to using the vMotion network for such replication. However, a separate network can be configured as well. Reads are not replicated since the backend storage appliance is always in-sync as far as reads go. In the event of a host failure, VirtuCache syncs the backend storage appliance from the write replicas on other hosts.

NB1: Do not use ‘Write-Back 2 Replica’ caching policy unless you have more than 24 hosts in a cluster and 20gbps replication network. Use ‘Write-Back 1 Replica’ policy for smaller cluster sizes.

NB2:  ‘Write-Back No Replica’ is our fastest caching policy since there is no network overhead incurred for write IO, however, this policy will result in data loss if a host fails. Despite this drawback, it is popular in Non-Persistent VDI, Dev/Ops, and Terminal Server VMs, where data loss can be tolerated.

Syncing dirty writes to backend storage.

VirtuCache has a background task that continuously syncs dirty writes to the backend storage. VirtuCache adjusts the speed at which dirty writes are synced based on the latency of the storage network and storage appliance, so as not to choke the SAN by trying to sync too quickly.

VirtuCache caps the maximum amount of dirty writes kept in cache to 2 minutes.

Typically only a few seconds worth of dirty writes will be kept in cache, since VirtuCache continuously flushes the dirty writes in cache to backend storage. However, if the storage appliance becomes very slow and/or there is a large burst of VM writes, VirtuCache will not be able to sync the dirty writes with the backend storage quickly, and so the dirty writes in cache will keep accumulating. Now, at no point in time will VirtuCache keep more than two minutes of dirty writes in local cache. This is so that VirtuCache can handle host failure gracefully. If a host fails, VirtuCache will immediately sync the mirrored copy of write cache (stored on other hosts in the cluster) for all the VMs on the failed host to the backend SAN appliance. VMware’s HA can only restart VMs on other hosts once this operation completes. Now we want all the VMs on the failed host to restart on other hosts in a maximum of two minutes and we also don’t want write replicas that are being flushed to backend storage to choke the network,  hence the maximum amount of dirty writes that VirtuCache will keep in the cache is 2 minutes of writes. The 2 minutes are calculated in terms of SAN speed, so if the SAN speed is, say, 100MBps, the 2 minutes of dirty writes work out to 12GB of data.