Energistics WITSML Product Certification Program Server Test Suite #1 Testing Procedure Introduction This document details the process by which WITSML server vendors can put their products forward for testing in the Energistics WITSML Product Certification Program with the roll-out of Server Test Suite #1. The first test suite builds on the self-certification process already in place. Vendors of WITSML server products that already participate in the WITSML Product Certification Program are encouraged to put their products forward for testing with Test Suite #1. Vendors of WITSML server products that do not currently participants in the WITSML Product Certification Program are encouraged to participate before initiating the testing process outlined below. Test Suite #1 is now available for testing as an optional part of the overall Product Certification Program. Energistics intends to make Test Suite #1 mandatory at year-end so that it will become required for new or renewed certification beginning in 2010. In the future, Energistics and the WITSML Special Interest Group plan to define and roll-out additional test suites both for server products and for client products. Link to example self-declaration document and form to be completed: http://www.energistics.org/posc/witsml_registered_and_certified_products.asp The scope of Test Suite #1 covers the following WITSML Objects and interface capabilities: GetVersion GetCapabilities Well Object Wellbore Object Log Object Trajectory Object Mudlog Object Test Suite #1 Testing Procedure The outcome of the certification testing will be determined by the response to the following test procedures: 1. Pre-requisite: The self-certification portion of the WITSML Product Certification Program must be completed before testing can begin. The testing of capabilities will be within the scope of server functions, data footprint, and restrictions documented in the production information form. 2. Pre-requisite: Results of Get Version Request The results shall match version of the Standards being tested. Initially, testing for Test Suite #1 will be limited to Version 1.3.1. (The test procedures will be updated as new WITSML versions are released. There are no plans to perform testing for Version 1.2.) 3. Pre-requisite Results of Get Capabilities Request. - The results shall match with the capabilities documented on the certification production information form. (See above.) Product Certification Program Page 1 of 6 Created: 3/17/09
4. Server Test Suite #1. Results of Test Queries as executed against the provided dataset. Except as noted in the pass-fail criteria, the replies from the queries shall return exact data as in original test dataset. (See below for details of dataset to be used and queries to be run.) Server Test Suite #1 - Test Query List and details and dataset are available in the Energistics WITSML SIG collaboration site for Energistics Members. Test Suite #1 dataset is presented in both WITSML 1.3.1 XML file format and as a graphical representation for log curves and mudlog details. Fig 1 (below) details the Time and Depth Extent of this dataset for reference:- The results from the testing will be fed back to the vendor as both WITSML XML file format documents extracted as the Energistics Server Test Tool runs the queries. If any failures are encountered, these will be highlighted in order that the vendor can act on them prior to submitting for re-testing. Administrative charges associated with testing / Testing Fees are deferred at this time for Energistics members. Fig 2 (below) graphically represents the data within the Trajectory Object of the test data set for reference. Product Certification Program Page 2 of 6 Created: 3/17/09
Points of Note / Clarifications 1. Test Suite #1 s Test Data will be made available in valid WITSML version 1.3.1 XML data files for download from the Energistics WITSML SIG collaboration site. a. If a server does not declare support for WMLS_AddToStore for an object then it is the responsibility of the vendor to load this data onto their server product that is undergoing testing. 2. The queries will only be run if the server declares support for the associated method. a. If a server declares support for WMLS_AddToStore for an object type then the corresponding add queries will be run as part of the test. b. If a server declares support for WMLS_GetFromStore for an object type then the corresponding get queries will be run as part of the test c. If a server declares support for WMLS_DeleteFromStore for an object type then the corresponding delete queries will be run as part of the test. 3. The queries will be run in the following order as appropriate: add, get, delete. a. For add queries, independent objects will be added first followed by more dependent objects. b. For delete queries, dependent objects will be deleted first followed by more independent objects. 4. Test Suite #1 s Test Queries will be run by / on behalf of Energistics using the Energistics Server Test Tool. Vendors are encouraged to run the test queries on their own test tools in preparation although final results will depend on those achieved via the Energistics Server Test Tool. 5. Vendors requesting certification testing will be expected to provide Energistics with a valid Web accessible URL to their Server Web Service Interface as well as a valid Username and Password. 6. Energistics will commit to running the tests and passing back the results to the vendors within 4 weeks of receiving the vendor request. Vendors will then have a period of six months and up to 6 consecutive attempts to achieve certification after which time results will be made public. 7. A query will be successful if it passes the criteria as defined in Pass-fail Criteria below. Product Certification Program Page 3 of 6 Created: 3/17/09
Test Procedure Details There are five elements that are used in running tests: 1. The server product ( SERVER ) 2. The test suite dataset ( DATASET ) 3. The test suite requests ( REQUESTS ) 4. The test driver tool ( TOOL ) 5. The test run responses ( RESPONSES ) For Server Test Suite #1, Energistics provides a DATASET, QUERIES, (reference) RESPONSES and TOOL. The DATASET, QUERIES, and (reference) RESPONSES are accessible and downloadable from an Energistics computing environment. The DATASETs are defined as an add query. The TOOL is installed to run on an Energistics computing environment via remote access. To run a Server Test Suite, a vendor supplies the vendor s WITSML Server Product. Testing can be run in four successive stages: 1. Acquiring the Test Suite. In this step: a. The vendor downloads the DATASET, QUERIES, and RESPONSES files and the TOOL remote access login information from Energistics. b. If the server does not declare support for WMLS_AddToStore for a specific object type, the vendor loads the information in the corresponding add query file. 2. Vendor Local Testing. This is a recommended but optional stage of testing. The vendor organization runs the tests internally using its own local testing TOOL. Energistics is not involved. a. Vendor sets up its own testing TOOL for use. b. Run all of the relevant REQUESTS through the TOOL. c. Compare the RESPONSES generated by the local TOOL with the reference RESPONSES. d. If the results are not 100% satisfactory, make adjustments and re-test. 3. Vendor Remote Testing. This is the stage where the vendor runs the tests using the Energistics TOOL via remote access. a. Vendor logs into the Energistics TOOL and configures the SERVER information in the TOOL. b. Run all of the relevant REQUESTS through the Energistics TOOL. c. Compare the RESPONSES generated by the Energistics TOOL with the reference RESPONSES. d. If results are not 100% satisfactory, make adjustments and re-test (or revert to Stage 2.) 4. Energistics Testing. This is the official testing stage. a. Send test request to Energistics together with remote access login information for the vendor s SERVER. b. Energistics runs all of the relevant REQUESTS through the Energistics TOOL. c. Energistics compares the RESPONSES generated by the Energistics TOOL with the reference RESPONSES, determining PASS or FAIL. d. Send the results obtained to the vendor. Pass-fail Criteria 1. When WMLS_AddToStore is supported, successful get queries shall return 100% of the data points matching the original dataset value except as otherwise noted. Not only must the WMLS_AddToStore query complete successfully, in addition the added object must be accessible for subsequent retrieval using WMLS_GetFromStore. Product Certification Program Page 4 of 6 Created: 3/17/09
2. When WMLS_AddToStore is not supported, successful get queries may return a subset of the original dataset values but the returned data shall match the original dataset values except as otherwise noted. 3. The object uid values may be different from the original dataset value. This includes parentage values such as uidwell. 4. The child uid values may be different from the original dataset value but, when WMLS_AddToStore is supported, best practice would retain the original values. 5. Returned numeric floating point values shall be within 0.01% of the original dataset value. This includes numeric values encoded into a string type (e.g., logdata). 6. Except for logdata, order is not significant in recurring elements. Queries Add Well 1 Add Wellbore 1 Add Log Depth Add Log Time Add Trajectory 1 Add Mudlog 1 Get All Wells Get Well Details Get All Wellbores Get Wellbores in Well Get Wellbore Details Get Logs in Wellbore Get Log Header Get Log Details Get Trajectories in Wellbore Get Trajectory Header Get Trajectory Details Get Mudlogs in Wellbore Get Mudlog Header Get Mudlog Details Delete Well Delete Wellbore Delete Selected Log Delete Trajectory Pass-fail Criteria.. Returns full list of wells accessible to user on server. Returns original details of a well within allowed variance. Returns full list of wellbores accessible to user on server. Returns full listing of all wellbores with specified parent well. Returns original details of a wellbore within allowed variance. Returns full list of logs with specified parent wellbore Returns original details in a log header within allowed variance. The curve mnemonic may be honored in a case insensitive fashion. If the semantic description of the mnemonic is not the same as the original then it shall be semantically consistent. Returns original details in a log within allowed variance. Shall conform to above criteria for log header. If WMLS_AddToStore is not supported then the log organization (number of Log objects generated) may be different and will be reported. Returns full list of trajectories with specified parent wellbore. Returns original details in a trajectory header within allowed variance. Returns original details in a trajectory within allowed variance. Returns full list of mudlogs with specified parent wellbore. Returns original details in a mudlog header within allowed variance. Returns original details in mudlog within allowed variance. about the well is no longer available via WMLS_GetFromStore. about the wellbore is no longer available via WMLS_GetFromStore. about the trajectory or related trajectorystations is no longer available via WMLS_GetFromStore. Selected log deleted without adverse effects to other Logs. about the trajectory or related trajectorystations is no longer available via WMLS_GetFromStore. Product Certification Program Page 5 of 6 Created: 3/17/09
Delete Mudlog about the mudlog is no longer available via WMLS_GetFromStore. At this stage generating new UIDs for WITSML objects loaded during the test will not be seen as a failure but will be reported as a warning to the vendor as well as being reported back to the SIG for comparison purposes. (See note on Dialect issues below) Dealing with Dialect related issues Within the current implementation of WITSML there are inconsistencies in how the standards should be interpreted that result in what are commonly called dialect issues. Providing that any vender seeking certification is able to demonstrate their implementation is compliant with the current version of the WITSML Standards, they will not be penalized for differences in detail of the datasets returned. It is intended, however, to use this information about dialect differences as formal input to the WITSML version change process in an effort to reduce these dialect differences in the future. Product Certification Program Page 6 of 6 Created: 3/17/09