Delphix enables Agile Analytics

Introduction

When it comes to enabling business analytics there are two main schools of thought.

1.    Specialized Analytics

There are several specialized Analytics or Warehouse Systems and appliances, examples include Teradata, Netezza, Exadata and Hadoop.

These systems require that data is routinely copied, synchronized and translated into their own specific data format for processing.  This often means that data has to be captured from production during short time windows but does mean that subsequent analytics processing is offloaded from production sources.

These systems suffer from having out of date data, they require substantial storage to retain their own large copies of data and a completely separate and often expensive and proprietary infrastructure.  Note there are not too many new proprietary analytics and warehouse systems or appliances being produced, many organizations are focusing on Hadoop which is seen as lower cost, more open and easier to scale.  However Hadoop still requires (at least 3) copies of the data and mechanisms to keep its data files in sync with source data.

An additional challenge is how do you secure that data if needs to be masked.

2.    Abstract Analytics

There are several Analytics Tool that abstract underlying data sources and connect directly to native databases and files, examples include Microsoft SQL Server Analysis Services (SSAS), Oracle Business Intelligence Suite and Tableau.

These systems can avoid referencing out of date data by directly connecting to production sources.  They also avoid the need to maintain a large physical copy of the data.

However running workload on production to prepare and refresh aggregations and analytics cubes is a strain on production that should be avoided.  Also any queries that require to be resolved by the underlying database using pass through will also place workload on production.

Using production directly also means there is no opportunity to secure the data through data masking.

Enter Delphix                          

Delphix can provision, production or masked, virtual copies of all the data sources necessary to support your analytics system.

Delphix for Specialised Analytics

In the simplest case Delphix can support Specialized Analytics systems by allowing all the data refresh activity to be driven directly off of virtual data copies.  This takes the pressure off of production, eliminates the time window constraint and also allows for the data to be masked before being synchronized to the Analytics system.

Delphix is ideal for this scenario since can curate all the source data necessary to populate your analytics system.

Delphix can feed your warehouse

Delphix for Abstract Analytics

However there is much more benefit available by using Delphix with an Abstract Analytics system.  You can enjoy the benefits offered by Microsoft SSAS, Oracle OBIEE, Tableau, etc which do not need a separate siloed proprietary copy of substantial data sets.

Delphix provides fresh copies of production data providing scalable up-to-date analytics

With Delphix you can also avoid the potential performance and security risks these systems can create when underpinned directly by production databases.   Abstract Analytics systems can run directly on Delphix provisioned virtual data copies.  This completely isolates the analytics workload from production.

The virtual databases underpinning the Analytics Engine can be refreshed from production independently or in synchronization.  You can also have multiple versions of the virtual data sets.

To improve performance the virtual databases can have schema changes applied, such as indexes, materialized views, etc. which better support the analytics engine requirements.

You can even scale out by leveraging as many copies of the databases as necessary to parallelize the query activity across as many servers as possible.  You can leverage underutilized server capacity to support your analytics processes.

Finally you could perform some or all of this activity on a cloud based Delphix deployment.

Summary

Delphix enables faster, fresher, scalable analytics whilst offloading production, avoiding a siloed investment and leveraging your existing database licenses and database server investments.

Delphix versus Oracle Snap Clone for Oracle Database Logical Recovery

Background

More frequently than we care to imagine, customers require to be able to recover quickly from scenarios where a database table or data is lost from production either through physical storage corruption or logical mistakes.  Due to the size and complexity of the environments these issues can take a long time to recover from.  Frequently we find that a customer's recovery time objective is challenging to meet.

Many customers use storage replication for HA and DR, however this does not protect against physical storage corruption nor logical mistakes.  Any such corruption would be replicated intact.  Corruption issues can be identified by application users but frequently they are identified much later during backups using RMAN block validation.

An Oracle standby database using database flashback still does not solve the problem because of the time taken to flashback a database and the administration overhead of managing flashback recovery areas.

Customers require a solution that can start up a clone or virtual copy database in minutes, from a specific point in time, to quickly source a database object that can be used to restore the production copy.

Most of the end-to-end recovery time is consumed locating, restoring and recovering the right version of the database.  It is an iterative process to find the most recent non-corrupted database copy.

Both Delphix and Oracle Snap Clone could be viewed as alternative solutions to implement Logical Recovery.

Key Requirements for Logical Recovery

There are three key requirements to implement a logical recovery solution.

1.    Synchronising with Source Database(s)

This solution requires that you have access to a continuous rolling window of backups since you will need to restore database objects (usually tables) from a version of the production database immediately prior to the corruption or fault occurring.  You want to restore the latest good copy of the database object(s).

2.    Provisioning Database Copies

The solution must also be able to provision a database copy to a point-in-time extremely quickly as this may have to be performed several times before the most up-to-date ‘good’ version of the object is located.

3.    Simplicity

The entire recovery process must be automated, easy to use and fast.  Recovery scenarios are stressful events and intuitive ease of use avoids artificial delays during the recovery process itself.

Oracle EM 12c DBaaS Snap Clone for Logical Recovery

1.    Synchronising with Source Database(s)

For continuous synchronisation with a source database Oracle requires a replication method to maintain an up-to-date Test Master from which all clones will be provisioned.

The replication method is typically an Oracle Data Guard physical standby.

The storage on which the standby is running is limited to Sun Storage, NetApp or EMC.  You can use any other storage as long as you have a server running Solaris to create and manage a ZFS file system on that storage.

Challenges

1.     Requires a licensed database server to run the standby database instance

2.     Enough storage for a full physical copy of the source databases

3.     Administration overhead managing the standby database, for example, synchronising nologging operations on the source database and physical changes including datafile addition and deletion

4.     Limited set of storage options

5.     Additional licenses are required if using 3^rd party, NetApp or EMC, snapshots capabilities 

2.    Provisioning Database Copies

Oracle Snap Clone can provision point-in-time clones of the Test Master from storage snapshots.

Challenges

1.     All target servers where clones are deployed must be licensed with

·       Oracle Database Lifecycle Management Pack

·       Cloud Management Pack

3.    Simplicity

Even customers who are familiar with Enterprise Manager may find that installing, configuring and managing DBaaS Snap Clone is non-trivial.

Delphix for Logical Recovery

1.    Synchronising with Source Database(s)

Delphix synchronises with source database seamlessly using an RMAN initial full backup and incremental forever backups including transaction log files.  All the ingested data is compressed to typically 50% of the size.  The compression saves storage for the source and virtual copies but more importantly improves the IO performance of virtual copies.

When we perform an incremental synchronisation (SnapSync) we can also validate the integrity of the database using validated synchronisation.  This will ensure that the backup copy we have can be used for recovery and the data is intact.  It also speeds up the process of provisioning a virtual copy.  The validated SnapSync could be run as frequently as required to support the recovery time objective.

Advantages

• With Delphix there is no requirement for a Standby Database
• Synchronisation is automatic regardless of any database changes (nologging, physical etc)
• Easy to define a retention policy for the span of time you want to keep backups
• Validated Sync can continuously check for corruptions and recoverability of the Delphix database copy
• No additional Oracle licenses required

2.    Provisioning Database Copies

Delphix can provision a virtual copy of a database in minutes to any point-in-time within the retention period of the source copy.  Provisioning can be performed via the GUI, CLI or API.  You can also provision as many copies as you want concurrently, from independent points in time to accelerate the time to restore service.

3.    Simplicity

                       

Delphix is easy to install, has automated discovery of source systems, once synchronisation is established with a retention period and one or more target servers are configured to provision virtual databases to there is very little monitoring or maintenance required.

Key advantages of Delphix over Oracle DBaaS Snap Clone 

1.     Easy to install and configure

2.     Complete automation of restore and recover process

3.     No requirement for multiple standby environments, avoiding setup, maintenance, monitoring, server capacity and licenses.

4.     Continuous validation and recovery of ingested backups as a further safeguard against corruptions and providing early detection of issues.

5.     Reduced Storage requirement

6.     Lower maintenance

7.     Minimal Licensing, with no additional licenses required if want to expand usage for provisioning non-production database copies

8.     More References