Taming the Chaos: Structured Approaches to Data Management for Standalone Geological and Geophysical Application Environments

Transcription

1 Taming the Chaos: Structured Approaches to Data Management for Standalone Geological and Geophysical Application Environments Solutions to maximize your assets. Landmark Software & Services

2

3 Taming the Chaos: Structured Approaches to Data Management for Standalone G&G Application Environments Author: Dale Blue, Global Practice Manager, Information Management Unlike today s standalone geological and geophysical (G&G) software environments where data management is essentially unstructured a number of more structured approaches to upstream data management enable asset teams to access information of known quality, share vital data and collaborate across technical domains while preserving hard-won knowledge and results for reuse in future projects. In the quest to find and produce oil and gas resources more efficiently, operators depend on a variety of specialized software applications for analysis, interpretation and visualization of digital exploration and production (E&P) data. In recent years, standalone workstation-based applications for geological, geophysical, petrophysical and reservoir modeling workflows have become increasingly widespread. E&P organizations initially bring in standalone packages to help meet a specific, urgent business need. Typically, standalone applications are chosen based on a specific functionality, price/ performance, or ease of use. Many share a similar underlying architecture. They have relatively simple data storage and access mechanisms designed to optimize performance for a single G&G professional working on a single project. Project data are stored either in a flat file structure on the user s hard drive, in a local relational database (such as Microsoft SQL) or in a shared network drive. Initially, no attempt is made to ensure consistency or compatibility among standalone applications from multiple vendors. As the asset, business unit or company grows, the number of such applications increases to support more workflows. However, as users and projects increase, data volumes accumulate and data management problems proliferate. Users spend more of their valuable time moving, modifying and manipulating data of decreasing or unknown quality. Leveraging data for more than one project, sharing information and collaborating with team members is extremely difficult. As the costs of unstructured approaches to data access and storage mount, operators seek more coherent methods of managing G&G applications and data. Standalone G&G applications are optimized for a single professional working on a single project. Since all project data are stored locally, it is difficult for users to share information or collaborate across disciplines. 1

4 They want a data management environment that allows asset teams to access high quality data, share information across domains and preserve results for reuse in future projects. After highlighting the risks and costs inherent in today s multi-vendor standalone application environments, this paper presents three potential ways of taming the data management chaos. We outline the variations, advantages and disadvantages of each alternative. Then we offer recommendations for basic and advanced implementations, based on our experience resolving upstream data management challenges. Risks and Costs of Standalone Application Environments To understand the risks to data and associated costs of current unstructured approaches to data management, consider how E&P professionals kick off a new project. Typically they look for a similar project already populated with data in approximately the same geographic area. They save it as a new project, delete data they do not need and update existing data with information acquired since the original project. Finally, they add more data to fill out the new project data set. Since all project data are stored locally, no data are shared among applications. To work with data from elsewhere, users must export that data in a neutral format, such as ASCII, and reload it from scratch. Then they begin making their own interpretations, producing results quite different than the original project. At this point, two projects with similar or overlapping boundaries now have significant differences between them. Inconsistent data values. The same wells, seismic surveys and other data are duplicated in multiple projects, but data values are often inconsistent due to ad hoc modifications and updates introduced to only one project. Missing data items. Adding new data to one project means current data are missing from others. Over time, it becomes increasingly difficult to determine the origin and validity of particular items. Multiple interpretations. Different interpretations of the same formations in wells and structural horizons in seismic begin to proliferate. Apart from project start dates, there are few ways of determining the relative quality of each interpretation. Loss of data and knowledge. Users often discard previous projects and begin again from scratch since they lack confidence in existing information. In the process, value-added knowledge and actual data may be lost. Repurchase of existing data. As projects and data proliferate, intelligent reuse of data becomes challenging or impossible. Original data sets may be lost or misplaced, requiring the company to repurchase data it already owns. Lack of collaboration. Because users cannot share data or interpretations, collaboration among team members is rare. Multi-domain projects take longer than necessary, limit the scenarios teams investigate or force them to settle for suboptimal results. 2

5 Nonproductive time. Without a structured approach to data management, high-value G&G users spend an inordinate amount of their time looking for data, moving and cleaning up data and incurring nonproductive time time not spent analyzing or interpreting data. Such wasted time typically ranges from 30% or 40% to as high as 70%. This alone may be the most compelling reason operators look for more structured approaches to G&G data management. Potential Structured Approaches to Data Management Given the disadvantages of standalone environments and unstructured approaches to data management, what alternatives are available to tame the chaos? Any structured solution must enable users to quickly access known, high quality project data, collaborate across domains and preserve results for the future. At a high level, there are three potential alternatives to today s multi-vendor standalone environments. Option 1: Migrating to a single-vendor application environment Option 2: Internally developing a custom, multi-vendor data management system Option 3: Adopting an integrated, commercial multi-vendor data management solution Each potential solution requires, at minimum, the implementation of two components: (1) a central or common G&G data repository, and (2) a consistent form of data transfer between multiple applications and the central repository. Consider the advantages and disadvantages of each alternative. Adopting applications from a single vendor that share a database designed solely for them could improve multi-user storage and sharing. But it would limit freedom of choice and fail to leverage existing investments. Option 1: Migrating to a single-vendor application environment One possible solution to data management problems inherent in multi-vendor standalone environments would be to migrate to a consistent single-vendor environment. In other words, begin replacing disparate applications from different suppliers with multi-domain applications from one vendor that also provides a structured data repository of some kind. Several E&P software providers today actually have such a solution a relatively unified set of applications with an optional database designed solely for those applications. Within this closed, multi-domain environment, consistent data transfer may exist, providing real improvements in data sharing, handling and storage. However, there are significant limitations. For one thing, if the optional data management system is relatively new, it may be immature or unproven. It may not 3

6 cover the full range of G&G data types required. It may lack robust utilities for data loading, quality control, user access and security. The biggest disadvantage, however, is that a database designed for just one vendor s applications will not exchange data with applications from other vendors. Adopting a vendor-neutral data management system capable of integrating software of multiple brands would be a much smarter approach. Without such a solution which does exist operators will no longer be free to adopt applications of their choice. Data will become locked in to one vendor, which is unacceptable even to small operators who value freedom of choice. Furthermore, a single-vendor database would fail to leverage an organization s existing technology investments. Fortunately, there are two additional options, each of which provides a true multi-vendor solution. Option 2: Internally developing a custom, multi-vendor data management system Some oil and gas companies develop their own multi-vendor data management system by combining selected commercial components with custom, internal components. There are advantages to developing one s own solution. It can be tailored rather precisely to existing applications, the organization s unique workflows and its specific business requirements. Data browsing, transfer and loading can be very efficient, depending on the combination of technologies. One component of a structured, multi-vendor solution is the common data repository. Alternatives include an open E&P data model that requires customization prior to deployment or a commercial data repository that comes with E&P data management tools and utilities out of the box. A. Common data repository. Two well-established alternatives are available today for the storage and management of upstream industry data. One is an open E&P data model from an industry consortium, such as the Public Petroleum Data Model (PPDM) Association, which requires custom development prior to implementation. For example, it must be wrapped with loaders, data QC tools, other utilities and connectors to move data to and from applications. The other alternative is a proven commercial, vendor-neutral E&P data repository, such as the OpenWorks system from Landmark, which comes with significant data management tools and capabilities out of the box. The range of data types covered by either of these structured approaches to data management is extensive. Either can be deployed as a common data repository for standalone applications from multiple vendors. The major difference lies in the amount of internal effort and expertise required to customize, install, connect and maintain the system. 4

7 At minimum, a common repository must provide adequate security so users can access data without inadvertently modifying its quality. The system must support concurrent, multi-user access so teams can easily work with the same data. It must handle multiple coordinate reference systems (CRS), so data loaded in one format can be converted to the CRS required by another standalone application. Data must be validated so users know it meets sufficient quality standards. Finally, users need access both to original data and project results. B. Consistent data transfer. Even if an operator implements one of these upstream data management systems, it is still necessary to transfer data into and out of existing standalone G&G applications. Again, a number of options are available. (1) A horizontal industry extract transform load (ETL) engine could be used to consolidate data from existing data stores in the central repository. However, generic ETL systems lack preconfigured connections for the unique G&G applications and data types operators want to integrate. Often a third party firm or consultant must create appropriate data connectors, which can be costly. One way to achieve consistent data transfer is for IT staff to write custom links using vendors APIs and SDKs. (2) Another option is for internal IT staff to write their own custom links. One simple method would be a programming level script that automates the manual data import/export process. However, this does nothing to resolve data redundancy, consistency, quality and collaboration issues. In the past, many large oil and gas companies developed efficient, hardwired, point-to-point connectors between standalone applications and databases using application programming interfaces (APIs) and software developer s kits (SDKs) from the various vendors. These proprietary links could, in fact, move data very quickly. But IT departments found it slow, painful and expensive to develop, support, maintain and upgrade them over time. Every new release of an application or database required a rewrite of the link, often from scratch, which operators found impractical and unsustainable. 5

8 Commercial E&P middleware provides a simpler and more sustainable way of connecting multiple standalone applications with the central data repository. (3) A third, more viable option would be to deploy a commercial E&P middleware platform specifically designed to transfer data to and from the most common standalone G&G applications. Such a system should automate data transfers and provide users easy access to data, ideally through their own application interfaces. No one wants to log into a separate data management tool. Users should be able to select data they want to preserve and publish it to the central database from within their own applications. One example of a widely accepted middleware system is TIBCO s OpenSpirit application, which provides preconfigured data connectors and copy tools out of the box, as well as regular updates whenever vendors release new versions of their technology. It also maintains connectors to both PPDM and OpenWorks repositories. There are potential advantages in developing a custom, multi-vendor data management system by combining various commercial and proprietary components. But the smaller the operator, the less likely sufficient IT resources or expertise would be available for such an undertaking. Even large oil and gas companies today are minimizing non-core activities such as IT integration. Option 3: Adopting an integrated, commercial multi-vendor data management solution The third structured alternative to today s data management chaos is to adopt a commercial multi-vendor solution that requires no internal customizations, enhancements or extensions. It should include off-the-shelf components, preconfigured to work together out of the box. An integrated, commercial multi-vendor system leverages an operator s investments in existing standalone applications, without requiring the purchase of new G&G software. Deploying a full commercial solution fits better with the IT expertise of smaller companies while laying the foundation for an enterprise-level solution for the future. Nevertheless, there are several issues operators need to be aware of. More sophisticated data management systems may require additional personnel and budget. However, adding a few more data management staff frees up G&G professionals to spend more time on higher-value analysis and interpretation. To ensure a uniform environment across the organization, it is necessary to establish and enforce data standards, governance and validation processes. This may require guidance from outside experts. It can also take substantial time and effort to clean up, establish quality control (QC) and load data into the new central repository. This, too, may require third-party services. 6

9 Basic components of an integrated, commercial multi-vendor data management solution would include: (1) the common data repository, (2) middleware and (3) expert data management service and support. A. Basic components. The two initial components of an integrated, commercial multi-vendor data management solution are: (1) a proven E&P industry data repository, and (2) middleware for G&G data transfer. Due to governance, data quality and other implementation issues, operators should also consider a third basic component, especially as the enterprise grows: (3) expert information management service and support. The unique characteristics of E&P data and the technical complexity of G&G applications often require operators to engage a service provider with considerable industry experience. In addition to strategic services around standards and change management, operators may need tactical services for data migration, QC, user training and support. Operators looking for immediate benefits from a structured approach to data management can begin simply by installing a robust data repository and data transfer layer and migrating data from one or two existing standalone applications. Connections to more applications can be added over time. Addition of three advanced components would begin to transform a basic data management environment into an enterprise-level solution: (1) more sophisticated data QC tools, (2) a project data archive and (3) a web portal for seamless access and federated views of live and historical information anywhere in the world. B. Advanced components. With commercial, off-the-shelf systems, it is possible to implement additional tools, growing the solution incrementally as the organization s needs expand. Three other commercial elements will transform a basic data management environment into an enterprise-level solution. 7

10 (1) More sophisticated data validation and QC tools empower data managers to provide better service to users. For example, even after implementing a central repository, without proper quality assurance, users may be concerned about the completeness or integrity of data. Data managers need to compare multiple data sets at once, detect missing or duplicated information and clean up increasingly large volumes of data. They need to visualize complex data of many kinds seismic, well logs, numerical data and text. Sophisticated commercial QC tools can divide the central repository into two separate partitions: one for standard corporate data of known quality, and one for project results saved by users. When users modify data in some way and save them to the results partition, other users can be assured no changes have been made to corporate data stored in the standard partition. Advanced validation tools help establish business processes and quality criteria that must be applied to all data before they can be loaded or published to the corporate repository. Adding these QC components to a growing enterprise data management system can improve user confidence and prevent the loss, corruption or unnecessary repurchase of data. (2) Incorporating a project data archive also enables an operator to store completed projects offline without losing access to valuable knowledge. Generic file system backups can only restore local data from recent activities. They do not provide any means of searching or seeing contents without reloading the project into an application. Without a project data archive, E&P professionals may find it too difficult to reuse data and interpretations from former projects, forcing them to start over from scratch. A good project archive must document projects at appropriate milestones and store them in neutral formats so they can be reconstructed later without requiring a specific software application or version. In addition to individual data items, an archive should save complete sets of elements used to justify a particular decision; for example, all the data used to evaluate an asset for a lease sale or calculate reserve estimates. Ideally, the system automatically creates web pages with snapshots of interpretations, lists of data and other vital information about a project. Thus users can view results online long after the project is taken offline. This additional component effectively preserves hard-won corporate knowledge. (3) One final element is a web portal that allows users to search, browse, view and access all data in the environment through a single familiar interface. Asset teams can perform Google-like searches across both live data in the central repository and historical data in project archives. The web portal can be implemented first in a local office, and then across similar data management environments in remote locations anywhere in the world. Federated views of multiple data stores give users and authorized partners greater awareness of all available information, encourage reuse and foster collaboration even across geographic boundaries. 8

11 Landmark s recommended solution for enterprise-level data management includes both technology and services. The Smart Way to Tame Standalone Data Management Chaos Having considered three potential alternatives to the chaos rampant in standalone environments today, we strongly recommend adopting an integrated, commercial multi-vendor data management solution. Smaller operators should keep it simple, starting with just the common data repository and data transfer layer, and adding more advanced components in stages as their experience and needs grow. All the components we recommend below have been seamlessly integrated and deployed widely in E&P organizations across the globe. The OpenWorks R5000 database incorporates an elegant solution for managing both corporate standard data and project results in the same system using an advanced security model. 1. Common data repository. For the central data repository, we recommend Landmark s OpenWorks R5000 database, the de facto industry standard for G&G data in oil and gas companies worldwide. While it has long been the underlying project database for Landmark s classic G&G applications, operators also use it as a shared, vendor-neutral repository for non-landmark applications. In addition to results data, the R5000 version incorporates an elegant solution for managing quality-controlled corporate standard data in the same system. An advanced security model allows data managers to flag every piece of information as either public or private, thereby differentiating between the two types of data. The OpenWorks database is built on an Oracle database, providing full concurrent multi-user access and user-level security. It incorporates many PPDM standards, but it packages them in a complete, commercially-supported product. We provide an open API and SDK, which some of our larger customers use to link proprietary applications they have developed in-house. The OpenWorks database comes with data management tools for loading and reformatting data, converting coordinate reference systems, as well as editing, deleting and inserting data. It also links efficiently to the Engineer s Data Model (EDM ) software, a Landmark data repository designed for the drilling and operations engineering domain. 9

12 2. Consistent data transfer mechanism. For the data transfer layer, one solution we recommend is TIBCO s OpenSpirit middleware platform. It uses our SDK to maintain a commercial connector to the OpenWorks R5000 database. It also provides preconfigured data connectors for most standalone G&G applications on the market today. 3. Advanced data QC tools. For data administrators interested in ensuring the highest possible data quality, we recommend Landmark s PowerExplorer software. It provides sophisticated tools for the validation, visualization, comparison and cleanup of data from multiple databases. PowerExplorer software can identify missing or duplicate data sets, determine whether data values are within specified limits and correctly populate all required data attributes in the corporate repository. Add-ons such as Advanced Data Transfer (ADT ) and Reference Data Manager applications can automate workflows based on company-specific business rules for cleaning and publishing data from the results partition to the standard partition without manual intervention. 4. Project data archive. Landmark s Corporate Data Archiver (CDA ) system provides an integrated solution for data archiving. Data managers can take snapshots of OpenWorks projects at any point in time, document those projects and store a rich set of metadata as HTML pages. The CDA system stores data elements in application-neutral formats, such as SEG-Y for seismic data, LAS for well logs, and ASCII for interpretation results. Data are bundled in the context of a particular project for browsing, retrieval and reuse in the future long after the original project has been altered or deleted. 5. Web portal. For user web portals, Landmark works with a variety of familiar and emerging horizontal portal systems. We also provide the unique Web OpenWorks software, which can integrate with any web portal to allow users to browse E&P data in OpenWorks projects. In addition, we provide all the services necessary for operators to deploy an effective G&G user portal within their existing company web portal. 6. Data management services. While all the technologies in our ideal solution are proven and relatively straightforward, we recognize that implementing any new information management solution requires careful planning, deployment, training and change management. Landmark provides a wide range of expert data management and consulting services to help ensure success and optimize return on investment. Our consultants have extensive experience in diverse E&P and IT systems, and deliver both hardware- and software-neutral services. 10

13 15 Benefits of Integrated, Commercial Multi-Vendor Data Management Operators struggling with the proliferating chaos and costs of unstructured approaches to data management in standalone workstation-based environments have three basic ways of taming the chaos. While switching to a single-vendor database could improve multi-user sharing, it would sacrifice freedom of choice and current investments in multi-vendor applications. Internally developing a multi-vendor system that combines commercial and custom components could solve immediate data issues, but could prove expensive and unsustainable over time. Implementing an integrated, commercial multi-vendor data management solution in stages, based on an organization s evolving business needs, can pay off in at least 15 ways. It can: Leverage existing investments in applications. Maintain freedom of choice going forward. Make use of current, off-the-shelf technology. Eliminate custom development and maintenance. Align with IT resources of smaller companies. Provide robust data management utilities. Ensure corporate data of known quality. Prevent unnecessary repurchase of existing data. Support multi-user access and security. Enable multi-domain collaboration and data sharing. Ensure more productive use of geoscientists time. Preserve both data and knowledge for the future. Lay a foundation for enterprise data management. Expand incrementally as needs, data and users increase. Effectively deploy locally, regionally or globally. Landmark provides a range of services that help clients maximize the use of technology assets. Our consultants deliver application implementation, deployment, onsite mentoring and education programs. In addition, innovative technologies, key industry partnerships and highly experienced domain experts allow Landmark Services to deliver solutions that optimize clients existing assets and enable anywhere, anytime collaboration. These services include intelligent operations solutions, IT/data management and cloud hosting services to support clients national or global workforces. For more information, contact your Landmark account representative or send an inquiry to Landmark@Halliburton.com. 11

14

15

16 Landmark Software & Services Halliburton. All rights reserved. Sales of Halliburton products and services will be in accord solely with the terms and conditions contained in the contract between Halliburton and the customer that is applicable to the sale. H /2012