Archive: Posts

Infinio’s Read Caching versus Virtunet’s Read+Write Caching Software

The biggest difference is that we cache reads and writes, Infinio caches only reads. Even in a read dominated workload, caching writes helps accelerate even the reads. The below article explains how this is so.

Then there are other differences as well like we assign caching policies at the datastore level with VM level overide, whereas Infinio caches at the VM level only. We cache both VM IO and VMware kernel IO whereas Infinio caches only VM IO. The implications of such differences between our two software are detailed below.

Infinio and we have been in the host side caching space for about the same time. We were founded in 2010, Infinio in 2011. But that’s where the similarities end.

Our two software are architected very differently.

  1. Infinio caches only reads (called Write-Through caching) whereas we cache both reads and writes (called Write-Back caching).

    Why caching writes is necessary?

    The main arguments from read caching vendors like Infinio, Atlantis Computing, and Datrium, for why read caching suffices is that most traditional IT workloads are read dominated and secondly they contend that by caching reads to in-host SSD/RAM, the storage appliance is freed up to better service writes. Both these statements are true, but they are not strong enough arguments to not cache writes. In fact not caching writes also slows down reads. Here is the reason. In all real world applications, reads and writes are interspersed on the same thread. Since Infinio does not cache writes, pending writes in the queue will back up even the read requests behind them that are on the same thread. So even if these reads by themselves could be accelerated by Infinio, because reads and writes are interspersed on the same thread the reads too are slowed down. Now keep in mind that if you run a synthetic testing tool like IOmeter you can configure reads and writes to be on separate threads, which will result in reads being accelerated independent of writes, but this is not a real life use case.

    Secondly, many storage appliances, even the older ones, do OK on reads but not on writes, so accelerating even small amount of writes disproportionately improves VM performance.

    Thirdly, it so happens that applications that are write intensive are also those that are of the highest value to the customer, as is the case with transaction processing, ERP, healthcare EMR, order processing, and such other applications. Simply put, entering purchase orders (write intensive job) is higher value than running a report on historical purchase orders (read intensive job).

    Hence both reads and writes need to be accelerated.

    Why do other vendors not cache writes?

    Because caching writes is much harder to do.

    Here’s why. Writes can be accelerated only if data is written to the in-VMware host SSD without synchronously committing the writes to the slower primary storage. This opens up the possibility of data loss if the local VMware host fails. To protect against such data loss, VirtuCache mirrors the write cache to one additional SSD on at least one additional host in the same VMware cluster, think of it as RAID-1 over the network. This requires that we cluster SSDs across hosts, which then lets us mirror and distribute the write cache across SSDs across hosts in the VMware cluster. Now if a VMware host were to fail, VirtuCache will immediately sync the backend storage appliance from the mirrored copy of write cache on another host in the cluster. By doing so VirtuCache protects against data loss in case of host failure. Such complicated architecture is not needed in Infinio or any other read caching software. Since in Infinio only reads are cached (copied) from backend storage to in-host caching media, if the local read cache is lost due to host failure, the cache can be easily rebuilt from primary storage, since those reads are available on primary storage anyways. As a result there is no risk of data loss if hosts fail, and consequently clustering and data protection strategy are not required, and so read caching is much easier to develop than read+write caching software. A side note – in this paragraph, reference to in-host SSDs can be replaced with in-host RAM if VirtuCache is configured to use RAM instead of, or in addition to SSDs.

  2. Infinio caches at the VM level, we do so at the Datastore level. It’s quicker for the Administrator to enable datacenter wide caching by selecting Datastores than VMs since Datastores are fewer in number.

    With Infinio, the administrator configures caching on a per VM basis. So if you have hundreds of VMs, that will be a lot of work. As shown in the first screenshot below, by default, we cache at the datastore level. Since the number of datastores is much less than the number of VMs, it is quicker to configure cluster and datacenter wide caching at the datastore level. With VirtuCache, all the VMs in the datastore inherit the datastore wide caching policy. Now you can exclude VMs from being cached or assign VMs a different caching policy than its datastore policy as shown in the second screenshot below.

  3. Virtunet Screenshot 1: In VirtuCache, assign caching policy at the Datastore levelVirtunet Screenshot 1: In VirtuCache, assign caching policy at the Datastore level.

    Virtunet Screenshot 2: In VirtuCache, ability to override Datastore level caching policy at the VM levelVirtunet Screenshot 2: In VirtuCache, ability to override Datastore level caching policy at the VM level.

  4. We accelerate VDI linked clones and VMware snapshot operations that Infinio doesn’t. This is because Infinio accelerates storage IO that originates in VMs only, we accelerate IO that originates in VMs and VMware Kernel.

    Since Infinio caches on a per VM basis, they can only accelerate storage IO originating within VMs. Now some important storage operations in VMware, like linked clones in VDI (where end user VMs are created (cloned) from a parent VM), and creating/deleting snapshots, do not originate in VMs, these operations originate in the VMware kernel. Also in the case of VDI or snapshots if the snapshot or linked clone hierarchy is many levels deep, storage performance gets worse as the hierarchy gets deeper, and so it is beneficial that snapshot and linked clone operations be accelerated.