Problems with Storage Snapshot based DaaS

Storage Snapshot Cloning Architecture

Customer issues with Storage Snapshot based Cloning

Many customers have mentioned several common problems with their existing storage snapshot based cloning solutions.

To make clones from storage snapshots you must have a full copy of the source database.   It must be a copy because you don’t want to create storage clones on your production storage array and potentially compromise performance.  It must be a full copy because that’s how storage snapshots work.

The major problem here is that you need a mechanism to maintain this full master copy on a different storage array.  Using database vendor specific replication technologies will require you to establish and maintain a variety of solutions if you use more than one database vendor, so most customers use storage replication.  However storage replication does not guard against physical or logical corruption making its way to the replica copy, that might have been avoided using the database vendor replication methods.  In addition we are now locked into the storage vendor for replication, snapshots and clones.

The next issue is that the performance of storage snapshots tends to decline quite rapidly.   Some customers are forced to periodically instantiate new master full copies just to restore performance.   So now we have maintenance, availability and a storage overhead issue.

Finally most storage snapshot solutions do not have automated maintenance workflows to cover the three main requirements for DaaS which are synchronization, governance and provisioning.   For many customers this makes their storage snapshot based cloning solution non-scalable from an operational and performance point of view.

How is Delphix Different

Ø  Delphix is storage agnostic

Ø  Delphix does not require a full copy of the source database(s).

Ø  Delphix automatically synchronizes with source database and retains a compressed de-duplicated copy.

Ø  Delphix can validate the integrity of the Delphix maintained copy.

Ø  Delphix works the same for all supported databases.

Ø  Delphix reduces the workload on the underlying storage by minimizing storage IO.

Ø  Delphix automates the entire DaaS workflow end-to-end.

Ø  Delphix provides self-service interfaces for operational and end-user teams.

Delphix as a smart NFS Server

Delphix AppData over NFS

Delphix is best known for its ability to automatically virtualize entire databases but Delphix can also be used to virtualize filesystem files and folders.  This capability is called AppData which is designed to virtualize application software or any data held in files and folders.

There are two methods of using AppData to synchronise with files and folders.  You can either have Delphix synchronise by periodically scanning a set of folders to capture updates or you can manage the updates yourself by using an AppData mount point.

Delphix can provision virtual storage and present that to a server as an NFS mount point, what we call a virtual mount point, vMount.  You can then copy data into that mount point and because the copied data is managed by Delphix, the data is catalogued, compressed and de-duplicated between copied versions.   Now that Delphix has one or more copies you can provision shared copies of any version of this data to another server over NFS or as a physical copy.

A common use case for this approach is to handle databases that Delphix does not yet natively support.  At the time of writing we support Oracle, SQL Server, SAP ASE (Sybase), Postgres, MySQL and DB2.

For example, Sybase IQ is one of the most commonly used columnar databases used for analytics.   Like many analytics environments it is not usually possible to expand the read capacity because the databases are by nature very large and the cost and time would make provisioning more copies to expand read capacity too expensive.

Using Delphix AppData the process to ingest, govern and provision copies of Sybase IQ are:

1.     Provision storage for IQ backups using a Delphix mount point (vMount)

2.     Restore an existing backup into the vMount, optionally rename.  Initially restore a full backup but subsequently restore either a full, incremental or incremental since full backup.

3.     Bounce the Sybase IQ residing on the vMount to check validity of the restore

4.     Take a Delphix snapshot of the vMount and optionally create a bookmark

5.     Provision a virtual copy of a chosen backup version over NFS to a separate target server

6.     Startup Sybase IQ on the target server running against IQ database files provisioned by Delphix

This method will allow you to save a set of discrete, compressed and de-duplicated versions of your IQ database in Delphix via an NFS mount.

Being a columnar database Sybase IQ gets great compression on its raw data which means the IQ database files are highly compressed.  Delphix will still get some compression on those backup copies, however the main benefits of using Delphix in this way are:

1.     Each subsequent restore is de-duplicated and hence only deltas are held for each backup version.

2.     Full read-write copies of the entire IQ database can be provisioned in minutes, expanding the read/analytics capacity.

3.     All the database maintenance activity on the source IQ database carries on independently of all the virtual copies.

4.     As and when new versions of the source IQ database are copied to Delphix users can refresh their own copies to the latest version or stick with the one they have if preferred.

5.     If desired, Delphix can provision other databases alongside Sybase IQ

Summary

Delphix can act as a very smart NFS server allowing access to private copies of almost any data of any size, near instantly with minimal storage or operational overhead.