Showing posts with label Delphix. Show all posts
Showing posts with label Delphix. Show all posts

Thursday, 22 December 2016

Delphix Explained, the Party Edition


Party Time 


It’s the office Christmas party season and we were invited to one hosted by our landlords and so met many people from different businesses.   These included accountants, employment lawyers, film producers, estate agents and wealth managers;  so quite a diverse bunch but one thing they had in common was at least we had an idea what they did.    The inevitable question we were asked was “What does Delphix do?”

What Delphix Does  


Delphix accelerates business software release cycles, data analysis and reporting to increase company revenues and margins.

Delphix does this by enabling access to and control of unlimited, versioned full data copies allowing Delphix customers to develop and test applications and report and analyze data in parallel.

What is the Problem? 


For business software, companies have substantial application development, testing and operational teams, sometimes running to 1,000s of individuals, dedicated to building, enhancing and maintaining software applications used by their customers and employees.   Companies’ revenue and profitability are increasingly reliant on these applications and their ability to continuously enhance them.

For business intelligence and reporting, companies have teams of data analysts using tools to glean insight from curated data.  All companies require to run increasing numbers of operational, compliance and ad-hoc reports.

All of these scenarios require access to full, up-to-date copies of production and secured production data.

Development and test teams require independent full copies of the application software and its underlying data to develop, test and debug changes.  In general provisioning full copies of the application software and test suites is not a major issue but full copies of databases is non-trivial and commonly companies adopt compromises such as shared copies, reduced size copies and synthetic copies which leads to bottlenecks, quality issues and extended or delayed release cycles.  This leads to a discrete release process rather than a continuous release process.  The root cause is the inability for all members of the development, testing and operational teams to have not just their own copies of these databases but multiple versions as well, as they already do with the application software.

Data Virtualization  


In the world of IT there are many examples of virtualization, including virtual memory, virtual machines, virtual networks, virtual services and virtual storage.   The primary benefits of virtualization are the ability to support increased concurrent workloads, increased control of workloads and higher utilization of physical resources.   Delphix is the first and only company to virtualize databases.

Delphix data virtualization allows all users who need full sized production or secured production database copies to provision these automatically through self-service.  Users can not only provision copies on demand they can have multiple versions of those copies and they can label, share, archive, restore any point in time copy of any version.

Delphix data virtualization super charges development, test, reporting, and analytics by enabling parallel activity on shared virtualized data copies.

Conclusion


Even the Estate Agent gets it.



Tuesday, 4 October 2016

Delphix reduces Oracle Database License Consumption

One of the line items on a Delphix return on investment calculation is reduction in database licenses.  In truth this is more of a cost avoidance benefit rather than an immediate saving but nonetheless customers understandably want to know how this is achieved.

Databases are usually licensed based on processor counts making a database processor an expensive resource.  The costs for these resources are constant and hence, rather like an aircraft that only earns money when it is flying, the aim is to keep database servers as busy as possible.  For cloud deployments the primary method to reduce costs is to release resources when not required, even temporarily.  The same rule should be applied to on-premise.

Common setups that consume license unnecessarily include

1.    Dedicated Test and Development Servers

1.1. Dedicated test servers locked into a project or team because of the difficulty in re-tooling it to switch workloads.

1.2. Regionally aligned development servers for geographically split teams, because of the difficulty in re-tooling it to switch workloads.

2.    Offload Production

Running CPU consuming processes on database licensed CPUs that could be offloaded elsewhere.  Production servers are a good example where any process that can be offloaded to a downstream server should be.

3.    Premium Cost CPUs

Running non-production workloads on premium costs servers.  Some processors attract a higher scale factor for license calculations so using the lowest cost processors (from a licensing perspective) on non-production database servers may mean running development and test on a different operating system.

4.    Premium Cost Hypervisors

Some hypervisors, notably VMware are not recognized as hard portioned servers meaning the entire database server must be licensed because the hypervisor ‘walls’ are not recognized.  Using a hypervisor that is recognized for hard partitioning can eliminate this issue, of example Solaris Containers on Solaris x86.

5.    Multi-tenant

Running many database instances consumes excessive database server capacity.  Multi-tenant consolidates consumption at the cost of additional licenses.

6.    Peak Sizing

Sizing for peak capacity on-premise.  Use cloud bursting to handle peak and license only for business as usual on premise.

7.    Inefficient Database Usage

Since database processors are expensive and application performance is always important, performance tuning can kill two birds with one stone, improve application performance and reduce license consumption.


Warning

Before proceeding it is worth mentioning that Database licensing is a very sensitive topic and any actions taken to reduce license obligations must be validated with the database vendor.


How can Delphix help?

Lets take a look at each of the scenarios above and examine how Delphix can assist.

1.    Dedicated Test and Development Servers

Solution (Part 1): Stop and Start Virtual Databases

With Delphix when a database (database server instance and data) is not required it can be shutdown, safe in the knowledge that the database can be restarted at any time with no loss of data.

This simple capability allows a given database server to be shared or time shared by more users or database workloads, increasing database server utilization is key to getting better value from your database licenses.

Two prime examples where this works particularly effectively include sharing database server resources across timezones, where teams can utilize the same server at different times of the day and end to end provision and tear down of test resources including databases for automated testing.

Solution (Part 2): Database Versions (branches)

Delphix allows an individual to work on more than one database version without consuming additional database server resources.  Delphix can switch between database versions, shutting down the current version and starting up the desired version.


2.    Offload Production

Solution: Delphix virtual databases

This is one of the primary goals of Delphix, to allow production servers to focus on supporting production activity and offloading any non-production or semi-production workloads to non-production servers and virtual databases.  For example spinning up a set of reporting servers can offload reporting activity from production.  Spinning up a set of virtual databases synchronised to the same point in time can offload production from ETL processes used to maintain Warehouse or Analytics databases.


3.    Premium Cost CPUs

Solution:  Cross Platform Provisioning (XPP)

Oracle uses a scaling factor for processor licenses depending on the processor in use and this ranges from 0.25 to 1.0.  This can have a substantial impact on license costs, see Oracle Processor Core Factor Table.

Delphix can be configured to ingest data from one Oracle platform and provision to another, for example from AIX to Linux.   This capability relies on Oracle transportable tablespaces and RMAN endian conversion but once configured becomes a completely automated, repeatable workflow.

All development and test activity can be performed on lower cost Linux environments and any AIX specific tests can be covered during pre-production testing on an AIX test environment.

4.    Premium Cost Hypervisors

Solution:  Heterogenous Provisioning

Several customers choose to run on Solaris x86 since Solaris Containers are treated as hard partitions allowing physical servers to be carved up into reduced size licensed virtual servers.

Conversion to Solaris x86 can be performed directly between Linux and Solaris x86 Oracle databases, and this capability will be available seamlessly from Delphix in an upcoming release.  Just as for XPP, a Pre- Production Environment can be used to test any platform specific issues after all the development and the vast majority of test iterations have been performed on Solaris x86.

5.    Multi-tenant

Solution:  full support for pluggable databases

Oracle’s 12c multi-tenant database option allows multiple, pluggable, databases to share the same set of background processes and memory reducing the overall CPU consumption,  The trade off is between the cost of the multi-tenant option and the alternative of licensing more CPUs to handle multiple standalone instances.

If it works for you, Delphix can virtualize and provision pluggable databases which can be assigned to individuals allowing them to manipulate their pluggable databases as if it is a standalone database.

6.    Peak Sizing

Solution:  Burst into the Cloud

Almost all customers size their non-production environments for peak activity on the assumption that all non-production activity will run on-premise.

With Delphix, synchronizing on-premise data with a cloud based Delphix Engine is trivial to setup and maintain.  Reducing the on-premise estate to business as usual activity and then leveraging cloud based environments for peak activity reduces your overall server and license spend.

7.    Inefficient Database Usage

Solution:  Automated performance tests

Delphix is perfect for running relentless performance tests, both regression tests and benchmark tests, particularly if these are automated.


Summary

Delphix is a key component for any strategy to reduce or at least trim expansion of database licenses and database server resources.   The range of options is:

1.    Workload Consolidation on development and test servers, through fast provision/de-provision
2.    Offload Production by supporting all additional workloads on non-production servers
3.    Lower Cost CPUs, through cross platform provisioning
4.    Lower Cost Hypervisor, through heterogenous provisioning
5.    Multi-Tenant, full support for Oracle 12c multi-tenant
6.    Trim on-premise Capacity, use Delphix to burst peak workloads into the cloud.
7.    Tune to Trim, run performance tests routinely and systematically with Delphix

With this range of options there is something for everyone.  However it is true that this kind of initiative must be planned and executed over time.



Wednesday, 4 May 2016

Delphix as a smart NFS Server



Delphix AppData over NFS






Delphix is best known for its ability to automatically virtualize entire databases but Delphix can also be used to virtualize filesystem files and folders.  This capability is called AppData which is designed to virtualize application software or any data held in files and folders.

There are two methods of using AppData to synchronise with files and folders.  You can either have Delphix synchronise by periodically scanning a set of folders to capture updates or you can manage the updates yourself by using an AppData mount point.

Delphix can provision virtual storage and present that to a server as an NFS mount point, what we call a virtual mount point, vMount.  You can then copy data into that mount point and because the copied data is managed by Delphix, the data is catalogued, compressed and de-duplicated between copied versions.   Now that Delphix has one or more copies you can provision shared copies of any version of this data to another server over NFS or as a physical copy.

A common use case for this approach is to handle databases that Delphix does not yet natively support.  At the time of writing we support Oracle, SQL Server, SAP ASE (Sybase), Postgres, MySQL and DB2.



For example, Sybase IQ is one of the most commonly used columnar databases used for analytics.   Like many analytics environments it is not usually possible to expand the read capacity because the databases are by nature very large and the cost and time would make provisioning more copies to expand read capacity too expensive.

Using Delphix AppData the process to ingest, govern and provision copies of Sybase IQ are:

1.     Provision storage for IQ backups using a Delphix mount point (vMount)
2.     Restore an existing backup into the vMount, optionally rename.  Initially restore a full backup but subsequently restore either a full, incremental or incremental since full backup.
3.     Bounce the Sybase IQ residing on the vMount to check validity of the restore
4.     Take a Delphix snapshot of the vMount and optionally create a bookmark
5.     Provision a virtual copy of a chosen backup version over NFS to a separate target server
6.     Startup Sybase IQ on the target server running against IQ database files provisioned by Delphix

This method will allow you to save a set of discrete, compressed and de-duplicated versions of your IQ database in Delphix via an NFS mount.




Being a columnar database Sybase IQ gets great compression on its raw data which means the IQ database files are highly compressed.  Delphix will still get some compression on those backup copies, however the main benefits of using Delphix in this way are:

1.     Each subsequent restore is de-duplicated and hence only deltas are held for each backup version.
2.     Full read-write copies of the entire IQ database can be provisioned in minutes, expanding the read/analytics capacity.
3.     All the database maintenance activity on the source IQ database carries on independently of all the virtual copies.
4.     As and when new versions of the source IQ database are copied to Delphix users can refresh their own copies to the latest version or stick with the one they have if preferred.
5.     If desired, Delphix can provision other databases alongside Sybase IQ



Summary

Delphix can act as a very smart NFS server allowing access to private copies of almost any data of any size, near instantly with minimal storage or operational overhead.