Delphix Explained, the Party Edition

Party Time 

It’s the office Christmas party season and we were invited to one hosted by our landlords and so met many people from different businesses.   These included accountants, employment lawyers, film producers, estate agents and wealth managers;  so quite a diverse bunch but one thing they had in common was at least we had an idea what they did.    The inevitable question we were asked was “What does Delphix do?”

What Delphix Does  

Delphix accelerates business software release cycles, data analysis and reporting to increase company revenues and margins.

Delphix does this by enabling access to and control of unlimited, versioned full data copies allowing Delphix customers to develop and test applications and report and analyze data in parallel.

What is the Problem? 

For business software, companies have substantial application development, testing and operational teams, sometimes running to 1,000s of individuals, dedicated to building, enhancing and maintaining software applications used by their customers and employees.   Companies’ revenue and profitability are increasingly reliant on these applications and their ability to continuously enhance them.

For business intelligence and reporting, companies have teams of data analysts using tools to glean insight from curated data.  All companies require to run increasing numbers of operational, compliance and ad-hoc reports.

All of these scenarios require access to full, up-to-date copies of production and secured production data.

Development and test teams require independent full copies of the application software and its underlying data to develop, test and debug changes.  In general provisioning full copies of the application software and test suites is not a major issue but full copies of databases is non-trivial and commonly companies adopt compromises such as shared copies, reduced size copies and synthetic copies which leads to bottlenecks, quality issues and extended or delayed release cycles.  This leads to a discrete release process rather than a continuous release process.  The root cause is the inability for all members of the development, testing and operational teams to have not just their own copies of these databases but multiple versions as well, as they already do with the application software.

Data Virtualization  

In the world of IT there are many examples of virtualization, including virtual memory, virtual machines, virtual networks, virtual services and virtual storage.   The primary benefits of virtualization are the ability to support increased concurrent workloads, increased control of workloads and higher utilization of physical resources.   Delphix is the first and only company to virtualize databases.

Delphix data virtualization allows all users who need full sized production or secured production database copies to provision these automatically through self-service.  Users can not only provision copies on demand they can have multiple versions of those copies and they can label, share, archive, restore any point in time copy of any version.

Delphix data virtualization super charges development, test, reporting, and analytics by enabling parallel activity on shared virtualized data copies.

Conclusion

Even the Estate Agent gets it.

Delphix enables Agile Analytics

Introduction

When it comes to enabling business analytics there are two main schools of thought.

1.    Specialized Analytics

There are several specialized Analytics or Warehouse Systems and appliances, examples include Teradata, Netezza, Exadata and Hadoop.

These systems require that data is routinely copied, synchronized and translated into their own specific data format for processing.  This often means that data has to be captured from production during short time windows but does mean that subsequent analytics processing is offloaded from production sources.

These systems suffer from having out of date data, they require substantial storage to retain their own large copies of data and a completely separate and often expensive and proprietary infrastructure.  Note there are not too many new proprietary analytics and warehouse systems or appliances being produced, many organizations are focusing on Hadoop which is seen as lower cost, more open and easier to scale.  However Hadoop still requires (at least 3) copies of the data and mechanisms to keep its data files in sync with source data.

An additional challenge is how do you secure that data if needs to be masked.

2.    Abstract Analytics

There are several Analytics Tool that abstract underlying data sources and connect directly to native databases and files, examples include Microsoft SQL Server Analysis Services (SSAS), Oracle Business Intelligence Suite and Tableau.

These systems can avoid referencing out of date data by directly connecting to production sources.  They also avoid the need to maintain a large physical copy of the data.

However running workload on production to prepare and refresh aggregations and analytics cubes is a strain on production that should be avoided.  Also any queries that require to be resolved by the underlying database using pass through will also place workload on production.

Using production directly also means there is no opportunity to secure the data through data masking.

Enter Delphix                          

Delphix can provision, production or masked, virtual copies of all the data sources necessary to support your analytics system.

Delphix for Specialised Analytics

In the simplest case Delphix can support Specialized Analytics systems by allowing all the data refresh activity to be driven directly off of virtual data copies.  This takes the pressure off of production, eliminates the time window constraint and also allows for the data to be masked before being synchronized to the Analytics system.

Delphix is ideal for this scenario since can curate all the source data necessary to populate your analytics system.

Delphix can feed your warehouse

Delphix for Abstract Analytics

However there is much more benefit available by using Delphix with an Abstract Analytics system.  You can enjoy the benefits offered by Microsoft SSAS, Oracle OBIEE, Tableau, etc which do not need a separate siloed proprietary copy of substantial data sets.

Delphix provides fresh copies of production data providing scalable up-to-date analytics

With Delphix you can also avoid the potential performance and security risks these systems can create when underpinned directly by production databases.   Abstract Analytics systems can run directly on Delphix provisioned virtual data copies.  This completely isolates the analytics workload from production.

The virtual databases underpinning the Analytics Engine can be refreshed from production independently or in synchronization.  You can also have multiple versions of the virtual data sets.

To improve performance the virtual databases can have schema changes applied, such as indexes, materialized views, etc. which better support the analytics engine requirements.

You can even scale out by leveraging as many copies of the databases as necessary to parallelize the query activity across as many servers as possible.  You can leverage underutilized server capacity to support your analytics processes.

Finally you could perform some or all of this activity on a cloud based Delphix deployment.

Summary

Delphix enables faster, fresher, scalable analytics whilst offloading production, avoiding a siloed investment and leveraging your existing database licenses and database server investments.

Delphix reduces Oracle Database License Consumption

One of the line items on a Delphix return on investment calculation is reduction in database licenses.  In truth this is more of a cost avoidance benefit rather than an immediate saving but nonetheless customers understandably want to know how this is achieved.

Databases are usually licensed based on processor counts making a database processor an expensive resource.  The costs for these resources are constant and hence, rather like an aircraft that only earns money when it is flying, the aim is to keep database servers as busy as possible.  For cloud deployments the primary method to reduce costs is to release resources when not required, even temporarily.  The same rule should be applied to on-premise.

Common setups that consume license unnecessarily include

1.    Dedicated Test and Development Servers

1.1. Dedicated test servers locked into a project or team because of the difficulty in re-tooling it to switch workloads.

1.2. Regionally aligned development servers for geographically split teams, because of the difficulty in re-tooling it to switch workloads.

2.    Offload Production

Running CPU consuming processes on database licensed CPUs that could be offloaded elsewhere.  Production servers are a good example where any process that can be offloaded to a downstream server should be.

3.    Premium Cost CPUs

Running non-production workloads on premium costs servers.  Some processors attract a higher scale factor for license calculations so using the lowest cost processors (from a licensing perspective) on non-production database servers may mean running development and test on a different operating system.

4.    Premium Cost Hypervisors

Some hypervisors, notably VMware are not recognized as hard portioned servers meaning the entire database server must be licensed because the hypervisor ‘walls’ are not recognized.  Using a hypervisor that is recognized for hard partitioning can eliminate this issue, of example Solaris Containers on Solaris x86.

5.    Multi-tenant

Running many database instances consumes excessive database server capacity.  Multi-tenant consolidates consumption at the cost of additional licenses.

6.    Peak Sizing

Sizing for peak capacity on-premise.  Use cloud bursting to handle peak and license only for business as usual on premise.

7.    Inefficient Database Usage

Since database processors are expensive and application performance is always important, performance tuning can kill two birds with one stone, improve application performance and reduce license consumption.

Warning

Before proceeding it is worth mentioning that Database licensing is a very sensitive topic and any actions taken to reduce license obligations must be validated with the database vendor.

How can Delphix help?

Lets take a look at each of the scenarios above and examine how Delphix can assist.

1.    Dedicated Test and Development Servers

Solution (Part 1): Stop and Start Virtual Databases

With Delphix when a database (database server instance and data) is not required it can be shutdown, safe in the knowledge that the database can be restarted at any time with no loss of data.

This simple capability allows a given database server to be shared or time shared by more users or database workloads, increasing database server utilization is key to getting better value from your database licenses.

Two prime examples where this works particularly effectively include sharing database server resources across timezones, where teams can utilize the same server at different times of the day and end to end provision and tear down of test resources including databases for automated testing.

Solution (Part 2): Database Versions (branches)

Delphix allows an individual to work on more than one database version without consuming additional database server resources.  Delphix can switch between database versions, shutting down the current version and starting up the desired version.

2.    Offload Production

Solution: Delphix virtual databases

This is one of the primary goals of Delphix, to allow production servers to focus on supporting production activity and offloading any non-production or semi-production workloads to non-production servers and virtual databases.  For example spinning up a set of reporting servers can offload reporting activity from production.  Spinning up a set of virtual databases synchronised to the same point in time can offload production from ETL processes used to maintain Warehouse or Analytics databases.

3.    Premium Cost CPUs

Solution:  Cross Platform Provisioning (XPP)

Oracle uses a scaling factor for processor licenses depending on the processor in use and this ranges from 0.25 to 1.0.  This can have a substantial impact on license costs, see Oracle Processor Core Factor Table.

Delphix can be configured to ingest data from one Oracle platform and provision to another, for example from AIX to Linux.   This capability relies on Oracle transportable tablespaces and RMAN endian conversion but once configured becomes a completely automated, repeatable workflow.

All development and test activity can be performed on lower cost Linux environments and any AIX specific tests can be covered during pre-production testing on an AIX test environment.

4.    Premium Cost Hypervisors

Solution:  Heterogenous Provisioning

Several customers choose to run on Solaris x86 since Solaris Containers are treated as hard partitions allowing physical servers to be carved up into reduced size licensed virtual servers.

Conversion to Solaris x86 can be performed directly between Linux and Solaris x86 Oracle databases, and this capability will be available seamlessly from Delphix in an upcoming release.  Just as for XPP, a Pre- Production Environment can be used to test any platform specific issues after all the development and the vast majority of test iterations have been performed on Solaris x86.

5.    Multi-tenant

Solution:  full support for pluggable databases

Oracle’s 12c multi-tenant database option allows multiple, pluggable, databases to share the same set of background processes and memory reducing the overall CPU consumption,  The trade off is between the cost of the multi-tenant option and the alternative of licensing more CPUs to handle multiple standalone instances.

If it works for you, Delphix can virtualize and provision pluggable databases which can be assigned to individuals allowing them to manipulate their pluggable databases as if it is a standalone database.

6.    Peak Sizing

Solution:  Burst into the Cloud

Almost all customers size their non-production environments for peak activity on the assumption that all non-production activity will run on-premise.

With Delphix, synchronizing on-premise data with a cloud based Delphix Engine is trivial to setup and maintain.  Reducing the on-premise estate to business as usual activity and then leveraging cloud based environments for peak activity reduces your overall server and license spend.

7.    Inefficient Database Usage

Solution:  Automated performance tests

Delphix is perfect for running relentless performance tests, both regression tests and benchmark tests, particularly if these are automated.

Summary

Delphix is a key component for any strategy to reduce or at least trim expansion of database licenses and database server resources.   The range of options is:

1.    Workload Consolidation on development and test servers, through fast provision/de-provision

2.    Offload Production by supporting all additional workloads on non-production servers

3.    Lower Cost CPUs, through cross platform provisioning

4.    Lower Cost Hypervisor, through heterogenous provisioning

5.    Multi-Tenant, full support for Oracle 12c multi-tenant

6.    Trim on-premise Capacity, use Delphix to burst peak workloads into the cloud.

7.    Tune to Trim, run performance tests routinely and systematically with Delphix

With this range of options there is something for everyone.  However it is true that this kind of initiative must be planned and executed over time.