Implementing the Data Management Continuum A Practical Demonstration Authors Steve Cooper: President, EnergyIQ Scott Schneider: Director, Volant Grant Monaghan: President, Perigon Solutions This document is the property of EnergyIQ, TGS, and Perigon Solutions and may not be distributed either in part or in whole without the prior written consent of these companies.
Contents 1. Overview... 3 Section I: Concept... 5 2. The Data Management Continuum... 5 2.1. Phases of the Continuum... 5 2.2. Source Data Loading... 6 2.3. IT QC... 7 2.4. Geoscience QC... 7 2.5. Geoscience Analysis... 8 3. Data Maturity... 10 3.1. Narrowing the Tramlines... 10 3.2. Measuring Data Maturity... 12 3.3. Assessing Data Maturity... 12 Section II: Implementation... 13 4. Components of the System Architecture... 13 4.1. PPDM Data Model... 13 4.2. PPDM What is a Well? Definition... 13 4.3. Data Objects... 14 5. Source Data Loading... 15 5.1. Standard Formats... 15 5.2. DropBox... 15 6. IT QC... 16 7. Geoscience QC... 17 8. Geoscience Analysis... 18 9. Benefits... 19 9.1. Business Benefits... 19 9.2. Technical Benefits... 19 2013. EnergyIQ, Inc. All rights reserved. -2-
1. Overview The Data Management Continuum defines the phases of the lifecycle of data from the automated collection and validation of as-is data to the presentation of workstationready data and the capture of resulting institutional knowledge. Any point along the continuum represents that maturity of data at that point in time. Figure 1.1: The Data Management Continuum Most companies don t make it past the first the first phase of loading the source data and, even then, this data is often incomplete, of unknown quality, and not easily accessible by the organization. Consequently, end users take it upon themselves to gather and validate data themselves which can lead to important decisions being made based upon incomplete or poor quality data and any knowledge gained is lost to the rest of the organization. The challenge facing companies, therefore, is how to load, integrate, and validate different data types from multiple sources and quickly make this available to the organization for analysis and reporting. Once the data is loaded, an effective interface must be available that enables the quality and availability of the data to be continually enhanced over the life of the well. Nowhere is this challenge more evident than in key unconventional plays where huge volumes of data are being generated through ever increasing activity and faster drilling cycles. Companies must manage this data to ensure basic regulatory compliance and operational efficiency while trying to deliver true competitive advantage by being able to perform in-depth analysis of the best data available. This paper describes how this challenge was met for one company by integrating commercially available software into an architecture based upon implementation of the Data Management Continuum. Looking to increase operational efficiency within their Bakken play, this large Independent approached three vendors with complementary skills in the E&P data management space to design and implement a solution to meet the needs of the business: EnergyIQ focuses on building master data to manage the most trusted data of an organization 2013. EnergyIQ, Inc. All rights reserved. -3-
TGS provides adapters to enable data to be moved and transformed between applications and master data stores Perigon Solutions delivers software that enables the management and visualization of subsurface wellbore data A collaborative agile development process was adopted to deliver a solution comprised of an enterprise class master data store, tools for data visualization, delivery and synchronization of business-ready data, and subscription services to structured and unstructured data. The result is a data management solution that gets used providing improved analysis, productivity, and the retention of institutional knowledge. 2013. EnergyIQ, Inc. All rights reserved. -4-
Section I: Concept 2. The Data Management Continuum The Data Management Continuum represents the phases of the data lifecycle from the automated collection and validation of as-is data to the presentation of workstationready data and the capture of resulting institutional knowledge. This is represented in Figure 2.1. Figure 2.1: The Data Management Continuum 2.1. Phases of the Continuum There can be any number of phases in the Data Management Continuum depending on how it is defined within a particular organization. The concept, however, remains the same in that it represents the stages involved in transforming source data into a state that it can be consumed by applications into information that has been subject to rigorous quality control and finally into knowledge through the capture of expert analysis; see Figure 2.2. 2013. EnergyIQ, Inc. All rights reserved. -5-
Figure 2.2: Data Transformation In the implementation that is being described in this paper, there are essentially four phases of the Data Management Continuum: Source Data Loading: This represents the as-is state of the data that is loaded without any transformation or validation. IT QC: This represents the as-is data after it has been transformed and validated through automated processes. Geoscience QC: This phase delivers information that has been enhanced to meet the specific needs of the business. It requires manual intervention to complete activities such as the splicing and de-spiking of log curves, cross-linking of reference codes, and the resolution of data quality issues through manual research. Geoscience Analysis: This phase involves the capture of the institutional knowledge of the organization. As enhancements and interpretations are made to the available information, this is captured back within the master data store and published to the organization as a whole. This is typically referred to as Workstation Ready data. 2.2. Source Data Loading This represents the first stage in the Data Management Continuum and is primarily an automated process. Data is loaded as-is so that it can be immediately available to the organization. Many different sources of data are typically loaded and the sources are usually ranked and presented to the end user in priority order. One of the main challenges at this stage occurs when the different sources of data do not match on the well identifier (which is usually the case). A clear set of definitions needs to be established for identifying a well and the components of a well and a corresponding naming convention adopted. The PPDM What is a Well? initiative provides an excellent structure for well definition and identification and was used for the initiative described within this document. 2013. EnergyIQ, Inc. All rights reserved. -6-
Figure 2.3: PPDM What is a Well? Definitions Data validation at this stage is provided by the database structure and constraints to ensure that data types are correct and that integrity is maintained. 2.3. IT QC The IT QC phase of the Continuum is intended to apply the necessary data transformations and validation rules to ensure that the data is application consumable. One example of a transformation that would occur at this phase is in the case of log data. The log file would be initially stored in its original format such as DLIS but then transformed into a standard LAS format for viewing and analysis. At this stage, validation is performed by Preventative rules that catch all nonsense data and make sure that the stored values are reasonable. For example, a set of rules would be applied to ensure that no depth on a Wellbore is deeper than the deepest depth of the Well. Any transformations and validation rules would be applied automatically. It is considered that this phase is the domain of data analysts and the IT group and that the end of the IT QC phase represents the point at which responsibility for data enhancement moves primarily to the business. It should be noted that most companies do not progress past the stage of data loading, transformation and IT based validation at the enterprise level. The conversion of data into information and knowledge occurs within applications but this is only sporadically captured within a master data store. 2.4. Geoscience QC The Geoscience QC phase requires a much higher degree of qualified end-user participation such as from a geo-technician. This might include processes such as 2013. EnergyIQ, Inc. All rights reserved. -7-
splicing log curves together to create a continuous log from the surface to the bottom hole; see Figure 2.4. It will also involve the application of much more rigorous Data QC Rules that require resolution by experienced personnel. At this point, validation is performed by Detective rules that assess values in context with other data. For example, a ground elevation may be reasonable and within tolerance for a region but may vary significantly when compared with a localized digital elevation model (DEM). Figure 2.4: Log Curve Splicing 2.5. Geoscience Analysis The geoscientist performs detailed analysis on the available data through a variety of interpretive applications. It is expected at this stage of the data lifecycle that they have trust in the maturity of the data and that they no longer need to spend significant time validating the content. When outliers are detected in the data, the question becomes whether this is truly an error in the data or whether it represents a physical anomaly to be investigated further. 2013. EnergyIQ, Inc. All rights reserved. -8-
Figure 2.5: Geoscience Analysis Once the geoscientist has completed analysis of the data and either added interpretations or upgraded existing data, a workflow needs to be followed to transfer this knowledge to the master data store. To be used, this workflow must be as simple as possible; past implementations have failed to meet expectations because they have placed too much of a burden on the end user to enter additional data and follow complex processes. This phase of the Data Management Continuum can never be considered complete as long as a company is active in the area. The information will continue to be analyzed in new ways and different interpretations will be generated; all of this knowledge needs to be captured back to the master data store. 2013. EnergyIQ, Inc. All rights reserved. -9-
3. Data Maturity The progressive enhancement of data across the continuum represents increasing maturity, not just in the data, but also in the way that it is consumed by the end-user. As data is transformed into information and knowledge, the users trust in the data is increased and they spend less time validating the content and more time in analysis. The maturity of the data is measured at a point in time along the continuum and can be used to identify areas of risk. Figure 3.1: Data Maturity Very few companies move past the IT QC stage of data maturity and, as a result, knowledge is lost to the organization resulting in missed opportunities and expensive re-work. 3.1. Narrowing the Tramlines One way to think of the data management continuum is as a constantly narrowing of the tramlines for acceptable data as we move from the as-is to the to-be status; see Figure 3.2. In this example, when the data is loaded, ground elevation can be any valid numeric value. The IT QC preventative rules narrow that down to a range of acceptable ground elevation values within the US (probably would be further narrowed down by State or County at this stage). The Geoscience QC detective rules would narrow the tramlines further possibly based upon a comparison with a localized DEM to determine valid elevation ranges within a field or a reservoir as an example. Finally, through Geoscience Analysis, the possible ground elevation range could be narrowed further based upon an analysis of data from other sources including logs, adjacent wells, site surveys etc. 2013. EnergyIQ, Inc. All rights reserved. -10-
Figure 3.2: Narrowing the Tramlines Another example to illustrate the concept of increasing data maturity is that of grain density within a target formation. Pure quartz has a density of 2.65 gm/cc; therefore, a very clean and mature pure quartz sandstone has a density of 2.65gm per cc e.g. a typical aeolian/dune sandstone. Preventative Rules: When loading grain density sample data to the well database such as from core data analysis, if nothing is known about the rock then the IT QC includes validation of the UOM (gm/cc) and then a catch all tramline data values rule to only allow values between 1.9 and 4.1 gm/cc. Detective Rules: If, however, the lithology is known to be sandstone the "Rock Literate" data values rules (buoys) could be greatly narrowed such that the minimum and maximum values are then set to 2.65 gm per cc plus or minus 10% The detective rules may be expanded further to find anomalies where the grain density may be correct but the sample depth may be wrong and erroneously appears at a depth coincident to a known dolomite limestone section (ie with a known average grain density of 3.1 gm/cc). Geoscience Analysis: With this foundation, the geologist can then commence reservoir analysis by narrowing the value ranges further to spot viable outliers that now focus 2013. EnergyIQ, Inc. All rights reserved. -11-
him/her on sections where there happens to be less pure quartz sandstone with more feldspar grains. From a reservoir understanding/rock properties perspective, derivative consequences of such lithology variants as these can substantially influence the hydrocarbon production opportunities or challenges. 3.2. Measuring Data Maturity As with the quality of data, it is important to be able to quantitatively define data maturity. This can be achieved in a number of different ways to reflect the key objectives of the organization. In the implementation described within this document, data was described within Data Objects and so data quality and maturity is measured in terms of the individual objects. These objects are then grouped together according to the workflows that depend upon them as illustrated in Figure 3.3. Figure 3.3: Measuring Data Maturity The first table provides a measure of data quality through the IT QC phase while the second provides a measure of which data objects have been upgraded to the Workstation Ready stage of the Data Management Continuum. 3.3. Assessing Data Maturity It is very important that the end user is able to assess the maturity of the data that they are accessing and that they trust that it is as represented. To this end the data maturity measurement values should be easily viewable and comprehensible. 2013. EnergyIQ, Inc. All rights reserved. -12-
Section II: Implementation This section details the solution that was implemented by the 3 application vendors to meet the requirements of the customer. 4. Components of the System Architecture It is not the intent of this section to describe the full architecture of the system in great detail but there are a few components that are important to understanding the overall solution. 4.1. PPDM Data Model The EnergyIQ TDM solution is built on top of the PPDM data model and, consequently, all of the master data is loaded into the PPDM structure. The model is comprehensive, covering 50 different subject areas, and is flexible so that it can be extended to meet the needs of the business without affecting the integrity of the model or breaking compliant software. Important features of the model include the ability to manage multiple versions of the same records and to composite the preferred source to a single record for presentation back to the end user. A comprehensive set of Reference and Alias tables enable master data to be maintained for the organization and provide a first stage of Data QC when they are populated appropriately. The PPDM model has become a de-facto standard for E&P data management and is gaining increasing acceptance throughout industry. As a result, more and more software companies are building applications that are fully PPDM compliant while service companies are enhancing their stable of expertise around this platform. All 3 vendors selected for the implementation phase of this initiative have extensive experience with the PPDM data model and fully support it within their application suites. 4.2. PPDM What is a Well? Definition It was very important to establish key definitions for a Well and the components of a Well that would be adopted by the organization early in the design phase of the solution. Too often the mistake is made of assuming that everyone is speaking the same language and this can result in significant problems with future integration initiatives. It was agreed with the company that the PPDM What is a Well? definitions would form the foundation for the data structure. This work was completed several years ago and has withstood numerous reviews and revisions to where it is now widely accepted within industry. 2013. EnergyIQ, Inc. All rights reserved. -13-
The Well, Wellbore, and Completion definitions, in particular, proved invaluable in establishing common understanding between the different disciplines involved in the planning, drilling, and completion activities associated with developing a lease. This understanding resulted in the creation of a Unique Well Identifier (UWI) that could be extended to uniquely identify Wellbores and Completions. This UWI is critical in terms of integrating data, applications, and workflows across the full well lifecycle. 4.3. Data Objects EnergyIQ has implemented a Common Object Model (COM) as the foundation for their TDM suite of applications to provide a layer of abstraction between the individual applications and the underlying data model (e.g. PPDM). This layer of abstraction enables the applications to access data within multiple data stores through a single API and without the need to constantly adapt to changes within the data stores. The COM is comprised of individual Data Objects that define and manage critical data required by the business in decision making workflows and processes. A Data Object is comprised of the set of related attributes together with the metadata and quality rules that apply to those attributes, as well to the object as a whole; see Figure 4.1. Figure 4.1: Data Objects There are no industry accepted standards for the structure and content of E&P Data Objects although it is anticipated that this will evolve over time. The approach for this initiative, therefore, was to define the structure of the Data Objects based upon a thorough evaluation of the workflows and associated data requirements of the business. The resulting Data Objects provide the backbone for communications between the different applications in addition to defining the Business and Data Quality Rules that are used to establish Data Maturity along the Continuum. 2013. EnergyIQ, Inc. All rights reserved. -14-
5. Source Data Loading In the current implementation of the management solution, source data is loaded through two primary techniques: Direct processing of standard formats Loading non-standard formats via Dropbox approach 5.1. Standard Formats Figure 5.1: Source Data Loading EnergyIQ provides standard loaders for a number of commercial data sources including IHS and Drilling Info. The data loaders run automatically and populate the PPDM data model according to industry best practices. 5.2. DropBox For non-standard formats, Perigon s ipoint solution provides a sophisticated database to enable data to be mapped into a common structure. Every time that a file of a certain type is copied into the corresponding folder, ipoint transforms the data into the common structure and then pushes this to the EnergyIQ Generic Loader. At this point, the Generic Loader takes over, performs any necessary UOM conversions, applies basic quality rules, and copies the data into the PPDM database. 2013. EnergyIQ, Inc. All rights reserved. -15-
6. IT QC During this phase of the Continuum, the ipoint loaders provide the functionality to convert different file structures into a standard format. This applies in particular to log data where the original log files are stored to the master data store but then converted to a standard LAS format for consumption by the end user applications. Once the original data is transformed, it is loaded into the PPDM data model via the EnergyIQ Generic Loader and a rigorous set of IT Rules are applied to the data. These IT QC Rules are defined within the corresponding Data Object. Figure 6.1: IT QC The IT QC Rules can also be run against the database as a whole on a periodic basis to check the overall quality. The results can be displayed for the database as a whole or at a more granular level, such as State, County, or Field. The results can also be displayed by Data Object to provide a detailed quality analysis by the categories that are most important to the business. 2013. EnergyIQ, Inc. All rights reserved. -16-
7. Geoscience QC In the Geoscience QC phase of the Continuum, a number of tools have been provided through the Perigon ipoint interface to enhance data. These tools include the ability to splice log curves together, the ability to de-spike curves, plus tools to compare similar data sets across multiple application stores with the master data store. These tools are targeted at the geo-technician who has the necessary level of training and expertise to be able to make context aware decisions about the data. Figure 1.7: Geoscience QC The data comparison tools are important as they provide the ability to systematically identify typical data problems such as duplicate wells and information that is out of synch between application data stores and the master data store. Once these problems have been identified, the tools are available to correct them through the UI and audit the history of the changes through the master data store. The TGS Envoy tools play an integral part in the data comparison and synchronization phase. These tools are able to read data from all of the different stores for display within ipoint and then write data back to the target stores applying the necessary conversions and ensure that best practices are followed. In this phase, the geo-technician will also be responsible for the resolution of data quality issues identified through the Rules Engine. 2013. EnergyIQ, Inc. All rights reserved. -17-
8. Geoscience Analysis From a solution perspective, this phase of the Continuum is primarily concerned with the capture of knowledge that is embedded within application data stores as a result of analysis by expert geoscientists. Figure 8.1: Geoscience Analysis The success of the knowledge capture workflow depends to a large extent upon the support and participation of the geoscientist to push enhanced data back to the master store. Significant efforts have been invested in making the process as simple as possible so that there is little burden on the end user in relation to the long-term benefits. For example, an enhanced log file can be uploaded to the master data store through a simple drag and drop mechanism. As can be seen from the illustration in Figure 8.1, the architecture of the solution is consistent throughout all phases of the Continuum. The TGS Envoy adapters are responsible for read/write activities with the applications, ipoint provides the sophisticated data visualization and analysis tools, and the EnergyIQ software handles all interactions with the PPDM master store. 2013. EnergyIQ, Inc. All rights reserved. -18-
9. Benefits There are numerous short-term and long-term benefits to implementing this solution. It is useful to consider both the business benefits as well as the technical benefits. 9.1. Business Benefits The primary benefit to the business is a reduction in business workflow cycle time and hence accelerated hydrocarbon production via: Immediate access to data of known quality and maturity Increased ability for geoscientists and engineers to collaborate and share results Elimination of the gap between user s active project data and higher quality master data. 9.2. Technical Benefits From a technical perspective, the solution implements the Data Management Continuum in a way that is flexible and sustainable. The implementation has enabled 3 vendors to collaborate to deliver a solution that combines best of breed software components with industry best practices in an open and extensible architecture. Other applications can be integrated into this environment through the Data Object interface, web services API, direct access to the underlying master data store or any of a number of other touch points. Additionally, maintaining the master data in the PPDM data model ensures the protection of organizational IP without long-term dependence on a particular application vendor. These benefits can be summarized as: Open database platform for data accessibility Best of breed commercial solutions providing long term stability and support Web services based interface around Common Object Model for extensibility 2013. EnergyIQ, Inc. All rights reserved. -19-