Objectivity-210x140
Use Case:
Location
San Jose, CA
Challenges
  • Overcoming poor performance, due to Objectivity's storage IO intensive environment
  • Upgrading to an All-Flash array was far more expensive than what Objectivity had budgeted for this project.
Benefits
  • VirtuCache was able to drastically increase throughput and reduce latencies, thus reducing average time to completion for their build-test cycle by 75%.
  • Objectivity was able to achieve better performance at a lower cost than an All-Flash array.
Improving Jenkins Performance

The Virtunet Difference

  • VirtuCache caching to a combination of host RAM and NVME was much higher performing than the All-Flash array they were evaluating.
  • The cost was much less as well, and it didn't require them to replace their existing array.

By caching hot data to in-VMware host NVME SSD and RAM, VirtuCache was able to improve the performance of Jenkins based Continuous Integration(CI) process, which in turn resulted in quicker build-test cycles.

Objectivity Analyzes Geospatial Data.

Objectivity is a software company in Silicon Valley that develops an object database to store and analyze large amounts of mapping data. They had recently moved to a Jenkins based CI process.

Continuous Integration (CI) Process is Write Intensive.

Continuous Integration (CI) tools like Jenkins, Electric Cloud, Django allow software development teams to quickly run automated tests whenever new code is pushed to the source repository. This allows developers to get quick feedback on their code with smaller increments of code added to the repository so that problems in new code can be detected and corrected faster than the legacy process of pushing large amounts of code to the repository less frequently.

As a result of increased build frequency and a larger number of jobs running simultaneously, such a continuous build process is write-intensive. Since this workload was running in a dense VMware cluster, it was very random as well. Random write-intensive workloads are especially stressful for any SAN infrastructure.

All-Flash Array vs. VMs on Local SSD Datastores vs. Caching to Host RAM + NVME.

To improve storage performance Objectivity looked at two options initially, one was to upgrade to an All-Flash array (AFA) and a second option was to move their datastores to host-based SSDs. An AFA for hundreds of TB of data was cost prohibitive. So then they decided to use host-based SSDs for configuration, build records, and artifact storage with the remaining data staying in their existing backend array. However, that led to reduced manageability because VMware’s vmotion, high availability, and DRS were now not possible for the data that was on local SSDs.

With VirtuCache, Objectivity could benefit from the speed of SSDs (in fact faster, since they would cache to host RAM), and without losing vMotion, HA, and DRS features in VMware.

VirtuCache Caching to a Combination of Host RAM and NVME SSD.

VirtuCache was configured to cache to a combination of 256GB host RAM and 3.6TB Samsung PM1725b NVME SSD installed in a PCIe slot in each host. When both RAM and SSDs are assigned to VirtuCache as cache media, RAM becomes Tier1 cache and SSD becomes Tier2 cache. VirtuCache then cached reads and writes to this combination of RAM + NVME SSD from Objectivity’s existing backend HP 3PAR appliance.

In performance tests comparing VirtuCache to the HP All-Flash array that Objectivity was evaluating, VirtuCache did better for two reasons. (a) VirtuCache was using in-host cache media which was not constrained by the network or storage controller. In the case of VirtuCache, the cache media was directly attached to the host motherboard over high-speed memory and PCIe bus. (b) Cache media assigned to VirtuCache (RAM + SSD) was much higher performing than the SAS SSDs in the HP AFA.

High Throughput, Low Latency, and Cheaper than All-Flash Array.

With VirtuCache caching to in-VMware host RAM and NVME SSD, Objectivity was now able to process over 500MBps throughput per host at under 10 ms VM level latencies for their Jenkins based VMs. And the cost of adding VirtuCache with RAM and NVME SSD to their existing VMware + SAN infrastructure was 1/4th of the cost of an all-flash array upgrade.

The time it took for Objectivity’s developers to run their build-test cycles was reduced to a fourth of what it was before.