Agile Copy Data Management

Delphix Agile Copy Data Management

Copy Data Management

Whilst there has been a gradual evolution in agile infrastructure (IaaS) and software stack provisioning architectures (PaaS), Delphix provides the missing piece in the agile architecture jigsaw by addressing Data as a Service (DaaS) through agile Copy Data Management.

Copy Data Management is generally defined as a solution which takes care of the curation, provisioning and security of all non-production database copies for Development, Analytics, Operations and Data Protection.

Networked Storage

Storage was the first component targeted by outsourcers to consolidate.  Moving from siloed, dedicated, Direct Attached Storage (DAS) to networked storage allowed overall storage consumption to be reduced by consolidating unused capacity and also allowed storage to be managed centrally.  However one of the downsides was that networked storage, typically Storage Area Networks (SANs), required a dedicated fibre channel data network to ensure performance for the high volumes of data traffic moving between database servers and the networked storage arrays.

In today’s world of agile infrastructure where resources are pooled, and allocated using virtualization technologies, having a dedicated network is an anachronism.   A dedicated resource means that unused capacity cannot be used by other workloads and the dedicated resource itself cannot expand or burst out to use capacity from elsewhere.

A move from siloed networks requires high bandwidth networks where capacity can be appropriately partitioned and allocated to accommodate multiple workloads. The throughput and capacity of Ethernet networks are evolving extremely fast allowing a single team of network administrators to manage all network traffic.  High throughput Ethernet allows the network itself to be a pooled resource with all the flexibility, agility and cost advantages that brings.

Storage Snapshots

Storage Snapshots have been around for many years and were originally designed to enable consistent backups of active database systems.  A backup takes a long time and most database systems cannot be suspended for the entire duration of a backup.  Storage snapshots allow a database to be queisced for a brief duration whilst the snapshot is taken.  The database can then be un-queisced allowing transactions to proceed and a consistent backup of the database can then be taken asynchronously via the snapshot.

Clones of snapshots can be taken to provision database copies. However supporting clones has an impact on storage performance and hence clones are usually driven off a full copy of production maintained on a separate storage server.  This mitigates the performance impact of clones on the production storage array at the cost of maintaining a replication method and additional storage.

Unfortunately the performance of cloned storage snapshots also degrades quite rapidly over time with IOPS dropping by up to 50% and latency spikes of 20ms.

Like most technologies storage snapshots are great when used for what they were originally designed for but start to disintegrate for non-primary use cases.

Virtual Data

Virtualising a shared resource is a pre-requisite for IT agility, which is why Delphix virtualizes data.  Delphix virtual data allows many users to share the same copy of data with the appearance of having their very own private, dedicated, full sized, read-write copy.  Delphix compression and de-duplication allow Delphix to store a sequential timeline of multiple database versions spanning long retention periods.  Delphix compression also increases performance by reducing physical IO requests and expands the effective size of the Delphix data cache which further reduces the number of physical IO requests.  These are two of the primary reasons why Delphix virtual data is often faster than purely physical storage based solutions.

Designed for Agility

Delphix itself is deployed as a Virtual Machine to leverage the easier deployment, management, scalability and HA of a hypervisor environment.  Almost every customer has skills in managing Virtual Machine environments.

Delphix is storage agnostic and leverages the flexibility and high performance of Ethernet to provision access to shared database versions over the network using NFS and iSCSI. 

The final key ingredient for agility is automation.  Any manual process involved in common workflows becomes a constraint and renders a solution unpredictable and non-scalable.  Delphix automates the entire data capture, curation, provisioning and versioning processes.

Access to Delphix database version control is available out-of-the box through self-service automation providing GIT like capabilities for databases of any size.

We call this Agile Copy Data Management.