Introduction to PIDX XML Transaction Standards

Transcription

1 Introduction to PIDX XML Transaction Standards Document ID: V2.0 Page 1 of 51 Introduction to PIDX XML Transaction Standards v1_0

2 Document Changes Version Date Version Authors Change Description 23 JAN Larry Dyer, Gary Strickland Created outline and content description 6 FEB Larry Dyer, Gary Strickland 1 st draft of section 1 20 FEB Larry Dyer, Gary Strickland 1 st draft of sections 2 Added references 6 MAR Larry Dyer, Gary Strickland Merged section 3 into section 2 Completed 1 st draft of section 2 Completed 1 st draft of section 3 Started 1 st draft of section 4 Added references Added terms to be defined to glossary Started index 15 MAY Gary Strickland Completed 1 st draft of section 4 Started 1 st draft of section 5 30 MAY Gary Strickland Completed 1 st draft of section 5 Started 1 st draft of section 6 Updated index 12 JUN Gary Strickland Completed first draft Edited document to define acronyms in each section Added illustrations 24 JUN Larry Dyer, Darren Ebanks, Rick Conner, Gary Strickland Edited the participants handout for completeness, accuracy, and readability 26 JUN Gary Strickland Reorganized major sections Added PIP sections 1 JUL Rick Conner, Larry Dyer, Darren Ebanks, Gary Strickland 7 JUL Paul O Shaughnessy, Gary Strickland Edited participants handout for completeness, accuracy, and readability Reviewed slide deck Completed Deployment Plan Completed Lessons Learned Reviewed deliverables Document ID: V2.0 Page 2 of 51 Introduction to PIDX XML Transaction Standards v1_0

3 Document ID: V2.0 Page 3 of 51 Introduction to PIDX XML Transaction Standards v1_0

4 Table of Contents 1 Introduction Purpose of the Introductory Course Intended Audience What this course will cover What this course will not cover Why this course is important Electronic Commerce Overview The Need for Standards Business Process Alignment EDI XML Overview of PIDX Business Process Alignment Standards RosettaNet organization and standards RosettaNet Strengths PIDX XML Transaction Standards Fundamentals of PIDX XML Transaction Standards Action, Signal, and Process Control Messages Action Messages Signal Messages Scope of Signal Messages Process Control PIPs The PIDX Message Components Preamble Delivery Header Service Header Payload PIDX Partner Interface Processes PIP Model Business Operational View Functional Service View Defining a Partner Interface Process PIP Specifications Trading Partner Implementation Guidelines Document ID: V2.0 Page 4 of 51 Introduction to PIDX XML Transaction Standards v1_0

5 6 PIDX and RosettaNet Differences XML Schemas vs. DTDs PIDX Partner Interface Processes (PIPs) Two-action PIPs PIDX Dictionaries Authorization Authentication Non-repudiation Non-repudiation of Origin and Content Non-repudiation of Receipt Strengths of the PIDX XML Transaction Standards Message Identification Message Authentication Message Authorization Non-repudiation Data Confidentiality Data Integrity Reliable Messaging Exception Handling Global Adoption and Support Business Advantages Lower Processing Costs Lower Bad Debt Expenses Improve Days Sales Outstanding (DSO) Improved Visibility Business Analysis Barriers to Adoption and Implementation and Solutions High Adoption and Implementation Costs Competing standards and technologies Next Steps Appendix A: Message Examples Appendix B: Error Codes Appendix C: References Appendix D: Glossary Document ID: V2.0 Page 5 of 51 Introduction to PIDX XML Transaction Standards v1_0

6 Index Document ID: V2.0 Page 6 of 51 Introduction to PIDX XML Transaction Standards v1_0

7 List of Tables Table 1: PIDX X12 EDI Transaction Sets (PIDX, 1997) Table 2: EDI startup costs Table 3: Differences between RNIF 1.1 and RNIF Table 4: Business Signal Error Codes List of Figures Figure 1: B2B data exchange processes sequence diagram... 9 Figure 2: Interrelationships of RosettaNet Specifications (RNIF2 Core Specification, p. 2) Figure 3: Content level validation activity diagram Figure 4: PIDX action message MIME components Figure 5: PIDX action message MIME components Figure 6: Send Invoice PIP use case diagram Figure 7: Send Invoice PIP activity diagram Figure 8: Send Invoice PIP functional service view sequence diagram Document ID: V2.0 Page 7 of 51 Introduction to PIDX XML Transaction Standards v1_0

8 1 Introduction PIDX (Petroleum Industry Data Exchange) XML Transaction Standards are a set of documents describing standard methodologies to exchange business documents electronically in the oil and gas industry. The current PIDX XML Transaction Standards (PIDX) were developed by the oil and gas industry in PIDX includes: PIDX XML Transaction Standards, version 1.0 Standards Master (Core specification) PIDX XML Transaction Standards, version 1.0 Implementation Guide (IG) Various versions of service content XML schemas Official interpretations of the standards by the PIDX Standards and Guidelines Committee The service content XML schemas have been updated periodically. PIDX standards are based on RosettaNet standards. The PIDX transport, routing, and packaging (TRP) parts of the standard are mostly compliant with the RosettaNet Implementation Framework, Core Specification, version 2.0 (RNIF2). 1.1 Purpose of the Introductory Course The oil and gas industry has over 10 years experience implementing the PIDX XML Transaction Standards, version 1.0. The collective experience of more than 200 operators, suppliers, and third party technology providers has demonstrated that some standards implementation projects have successfully been completed on time and on budget, while others have not. The course will discuss the PIDX XML Transaction standards, the challenges and barriers behind the inconsistent success of implementation projects, and how adopting companies can overcome those challenges and barriers. 1.2 Intended Audience Business and technical personnel of companies currently implementing the standards, those companies who plan to implement them, and the third party technology providers who plan to facilitate implementation projects will benefit from the presentations and discussions. 1.3 What this course will cover The presentations and discussions will focus on the following: 1. An overview of the PIDX standards, including RNIF2 2. Successful implementations approaches 3. Issues that increase the cost, schedule, and the risk of project failure 4. Discussions on reducing costs, schedules and risk of failure 1.4 What this course will not cover The presentation and discussions will not attempt to teach class participants the details of the PIDX standards or RosettaNet Implementation Framework, version 2.0. Participants do not need to have a Document ID: V2.0 Page 8 of 51 Introduction to PIDX XML Transaction Standards v1_0

9 detailed understanding of the standards. We will discuss the standards sufficiently to understand they how they work, what they do, and how to adopt and implement them to achieve company goals. 1.5 Why this course is important Implementing PIDX XML Transaction Standards can help companies reduce costs, days sales outstanding (DSO), and provide better transaction visibility. Over ten years of experience implementing the standards has shown that the standards can be adopted and implemented efficiently. Attending the class will enable participants to take back to their companies the knowledge that will help them avoid recurring challenges. 2 Electronic Commerce Overview Electronic Commerce (EC) can be defined in many ways. IBM defines electronic business as the transformation of key business processes through the use of the internet. (IBM p. 1) Using this definition, EC includes unsolicited s, business-to-consumer electronic store fronts like Amazon, online advertising in search engines, and the business processes used in the exchange of information using internet technologies. The PIDX XML Transaction Standards focus on the exchange of business information. They describe the sequence of activities and data to realize the exchange of business information over the internet. The information exchanged using the PIDX standards is usually in the form of common business documents, such as purchase and service orders, field tickets, and invoices. The sequence diagram in figure 1 shows some of the business documents that can be exchanged using the PIDX standards, and the sequence of their exchange. Figure 1: B2B data exchange processes sequence diagram Document ID: V2.0 Page 9 of 51 Introduction to PIDX XML Transaction Standards v1_0

10 The transformation of business processes began before the internet became popular. Banks and other businesses began authorizing the transfer of money and business data using proprietary file formats over telecommunications networks in the 1970s. In the 1980s organizations exchanged business data using telecommunications networks and electronic data interchange (EDI) standards. In 2002, the Petroleum Industry Data Exchange (PIDX) subcommittee of the American Petroleum Industry (API) published the PIDX XML Transaction Standards that were meant to replace EDI. While most oil and gas companies use the PIDX XML standards, some still use X12 EDI. 2.1 The Need for Standards Before standards like automatic funds transfer (AFT) and EDI were developed to manage the exchange of business data, companies used telecommunications networks to exchange business data in proprietary formats. This process worked well for those companies that did a lot of business together, but it was not an efficient way to exchange data. Data exchange formats had to be changed when a company upgraded or changed its business systems, and new one-off formats had to be agreed to when adding new trading partners. Businesses realized efficiencies exchanging data electronically, and they also realized that standards to guide the exchange would make the process more efficient. Accordingly, in 1982, the American National Standards Institute (ANSI) Accredited Standards Committee (ASC) released version 1 of the X12 EDI Standards. X12 EDI is used mainly in North America. Another EDI standard, Electronic Data Interchange for Administration, Commerce and Transport (EDIFACT), was developed by the United Nations, and is used in primarily countries outside North America. EDI standards enabled companies to decouple their proprietary backend enterprise resource planning (ERP) systems from the business data formats they needed to exchange with their trading partners. This decoupling allowed companies to upgrade and change their backend systems without changing the data formats used to exchange business documents with trading partners. This solved the biggest problem prohibiting widespread electronic data exchange. It did not solve all the problems: Data exchange was still expensive, arcane, and complex, and only large companies could afford the systems and personnel required to run and maintain EDI middleware. When XML became a World Wide Web Consortium (W3C) recommendation in 1998, many businesses believed that XML could be used to eliminate the complexity, expense, and mystery associated with EDI, and finally enable global electronic data exchange. 2.2 Business Process Alignment Exchanging business documents requires trading partners involved in a business transaction to align their business processes so documents can be processed successfully. Business process alignment applies to manual and electronic systems; the companies exchanging the data must understand how to process the data. For example, to execute a business process (BP) such as, send invoice to customer, both the supplier s and the buyer s processes need to be aligned to make sure the invoice sent and received is paid. In paper-based systems, the people responsible for sending an invoice must include information the customer needs to process the invoice, such as PO numbers, Authorization-for-expenditure (AFE) identification, field ticket data, and third party invoices. The invoice is then enveloped, and mailed. Document ID: V2.0 Page 10 of 51 Introduction to PIDX XML Transaction Standards v1_0

11 The customer receiving the paper invoice verifies that it conforms to their business rules, confirms that it contains all the information needed to process it, verifies that the expense was authorized, enters it into their accounts payable system, and finally, pays the invoice. The trading partners processes become even more complex when something goes wrong, such as missing information, or unexpected pricing. The same business processes that buyers and sellers execute to send and pay a paper invoice need to be executed when trading partners process invoices using B2B technologies. However, electronic processing presents problems that don t exist in the paper world. The data format required by a customer and included in the suppliers invoices can vary by the supplier that sent the document. This formatting issue isn t a problem when processing a paper invoice, because the person in accounts payable responsible for processing the invoice can look through it to find all of the information required for payment. This is a much more complex activity when executing the same process electronically. Additionally, the suppliers sending invoices must consider the disparate business rules and information requirements that vary by trading partner. While the alignment process is much easier now, it still presents challenges oil and gas companies need to overcome. 2.3 EDI EDI has been used to align business processes since the 1980s. EDI is defined as the application-toapplication transfer of data between organizations using a standard, structured, machine-readable format (Bort, 1998, p. A1-2-3). It was one of the first forms of electronic commerce extensively used in business, beginning in the 1970 s. Understanding EDI concepts is important to understanding current widely used B2B standards in the oil and gas industry because current standards are based on EDI concepts. EDI is the most common technology used in B2B transactions today, with the U.S. dollar amount of transactions equaling approximately that of all other B2B transaction technologies combined (Schneider, 2013, p ). While EDI is not used as extensively in the oil and gas industry as it was in the 1990s, it is still used in some oil and gas B2B transactions, and it is used more widely by oil and gas companies transactions with banks, transportation companies, and healthcare-related companies. There are several EDI standards used currently. The two most commonly used EDI standards are ASC X12, and UN/EDIFACT (United Nations/Electronic Data Interchange For Administration, Commerce and Transport). ASC X12 is used primarily in North America, and UN/EDIFACT is a global standard. The two standards define over 500 cross-industry transaction sets (business messages) between them. Each industry uses a subset of the transaction sets. The PIDX Implementation Guideline for EDI, 4 th Edition describes a compliant subset of the 300 plus ASC X12 transaction sets. The PIDX X12 transaction sets are listed in table 1: PIDX X12 EDI Transaction Sets. Transaction Business Document Version Set ID 810 Invoice Payment order / Remittance Advice Price / Sales Catalog Trading Partner Profile Request for Quotation Response to Request for Quotation 2040 Document ID: V2.0 Page 11 of 51 Introduction to PIDX XML Transaction Standards v1_0

12 850 Purchase Order Purchase Order Acknowledgment Ship Notice / Manifest Purchase Order Change Request Buyer Initiated Purchase Order Change Acknowledgment Request Seller Initiated Order Status Inquiry Order Status Report Functional Acknowledgment 2003 Table 1: PIDX X12 EDI Transaction Sets (PIDX, 1997) EDI works well for those oil and gas companies implementing EDI standards. Companies reported reduced administrative handling costs, reduced delivery time for invoices, reduced float on payments, eliminated data entry, reduced errors, improved data consistencies, and efficiencies in cash application. As an example, in a 1999 survey, 3Com reported the costs to process a paper order to be $38, compared to $1.35 per order using EDI. The company estimated the annual savings processing orders and invoices using EDI to be $750,000, and total estimated savings when considering the reduction of data entry errors, the efficiencies due to better inventory management and reduced delays in processing to be $1.3 million (Falk, 1999). Other companies report significant cost savings using EDI. In 2008, average cost to process a paper order was $37.45 in North America, while the same order using EDI was $23.83 (Aberdeen Group, 2008). While the EDI cost savings have been substantial for those companies implementing EDI solutions, not all companies implemented EDI solutions. In 2000, API-PIDX reported that 95% of Fortune 1000 companies were using EDI solutions, but that only 2% of non-fortune 1000 companies were doing so. Those companies that do not use EDI solutions cite the cost and complexity of EDI systems as the main reasons for not implementing. Table 2 illustrates typical startup costs for processing EDI transactions inhouse. EDI System Component Cost Hardware $20,000 - $100,000 EDI software, 1 st year licenses and support agreements $50,000 - $200,000 Technical expertise, 1 st year $70,000 - $500,000 Value Added Network (VAN) charges $24,000 - $100,000 Total 1 st year costs $164,000 - $900,000 Table 2: EDI startup costs Annual costs may omit the costs of hardware and software, but the licenses and support agreements continue. The personnel costs and VAN charges will also continue for in-house EDI processing, so the costs of in-house EDI processing are likely to be slightly less than the 1 st year costs. With the emergence of the internet and extensible markup language (XML), the oil and gas industry and other industries felt the B2B elements were in place to reduce the costs of using EDI to an acceptable level, and bring business process alignment to small and mid-sized companies as well as large companies. Document ID: V2.0 Page 12 of 51 Introduction to PIDX XML Transaction Standards v1_0

13 2.4 XML XML is a relatively new technology created in 1996 under the auspices of the World Wide Web Consortium (W3C). The XML design goals included the following: XML should be easily usable over the internet Programs processing XML should be easy to create XML documents should be clear and human-legible XML documents should be easy to create XML terseness is not an important consideration The W3C issued XML 1.0 Recommendation in 1998 and XML 1.1 Recommendation in The business communities embraced XML as the way to bring business process integration to all companies, not just large global ones. The simplicity of XML, the WC3 mandate of ease of use, the availability of low-cost XML processors, and flexibility promised that all trading partners exchange business documents directly into the other s ERP systems without programming. The only thing blocking universal business process integration was the lack of an industry-standard XML protocol. PIDX chartered the Complex Products and Services Task Group (Com.Pro.Serv) in 2001 to develop oil and gas industry standards for exchanging business documents using XML. PIDX published PIDX XML Transaction Standards, version 1.0 in February, Overview of PIDX Business Process Alignment Standards PIDX currently supports three messaging standards: 1. X12 EDI (Electronic Data Interchange) 2. PIDX XML Transaction Standards, version Applicability Statement 2 (AS2) X12 EDI has been a standard since the 1980s, and is still used by many oil and gas companies, mainly for exchanging business documents with organizations outside its industry. The PIDX XML Transaction Standards are the only one of the three standards promoted by PIDX. AS2 was offered as a low-cost alternative to the RNIF2 TRP. They were used by a few oil and gas companies for a few years. Currently they are not used. According to the PIDX Standards Master, version 1.0 (PIDX1), the purpose of the standards is to promote standard processes for non-catalog, configurable products and services using ebusiness technology. The standards defined public processes, a transaction, routing, and packaging (TRP) protocol based on the RosettaNet Implementation Framework Core Specification, version 2.0 (RNIF2), and an XML business vocabulary based on previously defined XML vocabularies including the Chemical Industry Data Exchange (CIDX), xcbl, and OFS-Portal. To understand the PIDX XML Transaction Standards, one must understand RNIF2. RNIF2 describes the TRP protocol that enables participating supply chain members to implement RosettaNet partner interface processes (PIPs). PIDX did not adopt any of the other RosettaNet standards, including RosettaNet PIPs, or the RosettaNet Dictionary Framework. Document ID: V2.0 Page 13 of 51 Introduction to PIDX XML Transaction Standards v1_0

14 The PIDX exchange protocol relies on RNIF2, and it is mostly compliant with RNIF2. Oil and gas companies that use RNIF2-compliant messaging applications should not have problems adopting the PIDX standards. 3.1 RosettaNet organization and standards RosettaNet is a non-profit consortium of more than 500 technology, electronic components, and semiconductor manufacturing companies. RosettaNet was founded in the U.S. in 1998 to develop standards to automate inter-company supply chain business processes. In 2002 RosettaNet merged with The Uniform Code Council, which was renamed to GS1 US. GS1 US manages the RosettaNet standards. The RosettaNet standards include the core specification, a set of partner interface processes (PIPs) applicable mostly to the technology and electronics industries, and a dictionary framework defining information items in the standards. The core specification of the RosettaNet standards describes the TRP methodology. RosettaNet supports two versions of the core specification: RosettaNet Implementation Framework, version 1.1 (RNIF1), published in 1999, and RosettaNet Implementation Framework, version 2.0 (RNIF2), published in Both versions are still supported by RosettaNet, and both are free and open to the public. The major differences in the two core specification versions are summarized in table 3. Feature RNIF 1.1 RNIF 2.0 Transfer protocols HTTP HTTP, SMTP Attachments No explicit support. Required Provides for formal framework for attaching Private agreements. supporting documents to the service content. Service content and service header encryption Support for delivery to intermediaries Exception handling Not available. Not available. Described in a Technical Advisory issued separately. Recommends use of S/MIME based content enveloping scheme for encrypting the Service Content and also the Service Header. Adds the Delivery Header and makes recommendations when messages are sent through intermediaries. Integrates exception handling and message flow into the specification. Table 3: Differences between RNIF 1.1 and RNIF 2.0 RosettaNet goals focus on improving supply chain interactions between trading partners by specifying standards to align trading partner business processes. The standards specify what should be exchanged and how the exchanges should be executed. The standards apply only to public processes; those processes that involve packing and unpacking a RosettaNet message, and interactions between trading partners. Private processes are out of scope of the RosettaNet standards. Private processes include processes exporting and importing data from and to an ERP system, and those translating data to and from service content. Document ID: V2.0 Page 14 of 51 Introduction to PIDX XML Transaction Standards v1_0

15 The RosettaNet standard documentation includes the core specification (RNIF2), a dictionary framework, and documentation describing PIPs. The relationships between the RosettaNet documentation are illustrated in Figure 2. Figure 2: Interrelationships of RosettaNet Specifications (RNIF2 Core Specification, p. 2) The partner interface process (PIP) is the operational focus of the RosettaNet standards. The core specification, the business dictionary framework, service content DTDs, header DTDs, message guide lines, trading partner agreements, and associated private business processes support the execution of a PIP. Companies implementing RosettaNet-based standards must understand this: The successful execution of a PIP is the primary business and technical focus of the RosettaNet standards RosettaNet Strengths Many companies have adopted and implemented the RosettaNet standards. The standards offer a robust and freely available way to securely integrate specific business processes of independent organizations using disparate systems. The standards do not describe just the data to be exchanged between trading partners; they define specific processes, as well: What to send When to send How to send How to verify receipt Document ID: V2.0 Page 15 of 51 Introduction to PIDX XML Transaction Standards v1_0

16 How to secure How to verify processing status 3.2 PIDX XML Transaction Standards The PIDX XML Transaction Standards are modeled after the RosettaNet standards. The PIDX standards: 1. Use the RNIF2 standard for transport, routing, packaging (TRP), message reliability and security 2. Use the RosettaNet concept of the Partner Interface Process (PIP) 3. Rely on RosettaNet Message Guidelines to define the message header XML documents The RosettaNet standards were developed for the electronics and related industries. While the PIDX XML Transaction Standards, version 1.0 are based on the RosettaNet standards, there are a few, important differences. Some of the differences are necessary because of the needs of the oil and gas industry. Others restrict and change the RosettaNet core specification, RNIF2. 4 Fundamentals of PIDX XML Transaction Standards A basic understanding of the RosettaNet Standards, how PIDX restricts those standards, and the processes and general data is necessary to successfully manage implementation projects. This section discusses the types of PIDX messages, their components, the processes used to create, deliver, receive, and process a PIDX message, and the PIDX XML Transaction Standards documentation. 4.1 Action, Signal, and Process Control Messages There are two types of messages exchanged in a PIDX partner interface process (PIP): business action messages and business signal messages. Business action messages contain business documents such as invoices and purchase/service orders, and field tickets. Business action messages can also contain optional non-xml attachments. PIDX business signals are positive and negative messages that acknowledge the receipt of a business action message, and return the validation status of the business action message. The trading partner that received the business action message MUST return a business signal when the receiving systems can determine that the action message was sent by a trading partner authorized to send it. RNIF2 contains one positive and one negative business signal. Only business action messages are acknowledged; business signals are never acknowledged. RosettaNet defines a third message type called a process control message. A process control PIP is used to communicate process status outside of the context of the current process instance. PIDX XML Transaction Standards do not mention process control PIPs. Process control is included in this section for completeness Action Messages A PIDX action message contains a business payload. The payload MUST contain a business document formatted in accordance with a specific PIDX XML schema, such as an invoice, field ticket, or purchase order. The payload of an action message MAY contain other non-xml-formatted content, as agreed to by the trading partners implementing the PIDX PIP. Document ID: V2.0 Page 16 of 51 Introduction to PIDX XML Transaction Standards v1_0

17 4.1.2 Signal Messages RosettaNet signals are messages used to communicate the status of processing events of an action message in a receiver s system. Events that MUST be reported are base-level validation events: 1. The receipt and successful base-level validation of a message (Receipt Acknowledgment) 2. The receipt of an out of sequence message (Exception with a type of General Exception ) 3. The receipt of a message that has invalid grammar; it did not pass base-level validation (Exception with a type of Receipt Acknowledgment Exception ) Optionally, trading partners MAY agree to validate XML service content against the receiving trading partner s business rules. This is called content validation. Section Scope of Signal Messages, discusses base-level validation and content level validation. The PIP specification determines if and when a signal is returned to the sender of an action message. Currently, all PIDX PIPs require the receiver of an action message to return a signal. Receipt Acknowledgment Signal A receipt acknowledgment (RA) is a positive business signal that acknowledges the receipt of a valid action message. Validity of the action message is determined by base-level validation and (optionally) by service content validation requirements negotiated between trading partners. Exception Signals An exception signal is a negative acknowledgment message indicating an error in the structure or the syntax of the associated action message. The following exception types indicate the types of exceptions thrown in the receiver s system: Receipt-Acknowledgment Exception: a negative acknowledgment of receipt of a business action message sent when the action message is not structurally or syntactically valid General Exception: a negative acknowledgment signal sent to indicate an exception other than a structurally or syntactically invalid message The trading partner receiving a business action message MUST acknowledge the receipt with one of the signals discussed. However, RosettaNet recommends that authentication or authorization failures should not be acknowledged to minimize security risks. Note that if an exception is thrown before the receiving trading partner s system can identify the sending trading partner, RosettaNet standards prohibit sending a signal Scope of Signal Messages A receipt acknowledgment or exception business signal is sent when a trading partner receives a PIDX action message. If the action message was base-level validated, the applicable syntax and structure rules are satisfied. In addition to validating the action message s structure and syntax, PIDX allows trading partners to optionally agree to validate the service content of the action Document ID: V2.0 Page 17 of 51 Introduction to PIDX XML Transaction Standards v1_0

18 message against the receiver s business rules prior to sending the business signal. If the trading partner receiving the action message validates the service content before sending the receipt acknowledgment/exception, then the trading partner MAY return an exception if the service content is not valid. The exception type is General Exception and the error code to use is PRF.DICT.VALERR. The activity diagram in figure 2 shows the process and message flow when the service content of an invoice is validated before creating a receipt acknowledgment or exception signal. Figure 3: Content level validation activity diagram Base-level Validation Rules The four XML parts of a PIDX message (preamble, delivery header, service header and service content) MUST be validated against a RosettaNet DTD (for the three header documents) or schema (for the service content). The following is the minimum level of validation that is required on each of the XML body parts. Validation against these rules is called base-level validation: 1. The XML document MUST be compliant with its corresponding DTD or schema 2. Where an element s data type and/or length is specified in the corresponding schema (in the case of the service content) or Message Guideline (in the case of the header parts), the element MUST be validated against the schema or Message Guideline 3. In validating header documents, where the cardinality specification of an element in the Message Guideline is different from the corresponding specification in the DTD, the specification in the Message Guideline is more accurate and MUST be adhered to 4. In validating header documents, where the sequence or naming of an element in the Message Guideline is different from the corresponding specification in the DTD, the specification in the DTD is more accurate and MUST be adhered to If a message does not follow one or more of the above rules, then it MUST be deemed invalid Process Control PIPs RosettaNet defines process control PIPs to communicate process states outside of the context of the current process instance. For example, a new instance of the 0A1 Notification of Failure PIP is started when exceptions happen under a specific condition (namely, when the process is in execution state at one partner s system and may have possibly reached a completed state in the other partner s system) during the execution of any other process. For example, assume that a customer sends an invoice response action message in a single-action invoice PIP to a supplier. Assume further that the supplier validates the invoice response action message and returns a receipt acceptance to the supplier, completing the PIP for the supplier. If the customer subsequently discovers issues that prevent payment of the invoice, the 0A1 PIP should instantiate, sending a process control message to the supplier indicating the process was terminated in an exception state. Current PIDX standards do not address process control PIPs. Document ID: V2.0 Page 18 of 51 Introduction to PIDX XML Transaction Standards v1_0

19 4.2 The PIDX Message Components A PIDX message is a MIME multipart/related message consisting of three XML headers (preamble, delivery header, and service header), and a payload that includes mandatory service content, and optional attachments. The purpose of the headers is to enable the trading partner and any intermediaries to Identify the message as a PIDX-compliant message (preamble) Identify the sender for authentication and authorization (delivery header) Identify the context of the message (service header) Figures 4 and 5 show the MIME components of a PIDX message. The payload includes the service content, and in the case of action messages, zero or more attachments. The payload of a signal message consists only of the service content. The service content is always a PIDX-compliant XML document. The headers and payload are packaged together using a MIME multipart/related construct. Trading partners may optionally agree to digitally sign the PIDX message. The PIDX standards use XML schemas to define message service content. The standards use RosettaNet DTDs to define XML header parts. Figure 4: PIDX action message MIME components Document ID: V2.0 Page 19 of 51 Introduction to PIDX XML Transaction Standards v1_0

20 Figure 5: PIDX action message MIME components Preamble The preamble header identifies the standard and version with which a PIDX message must comply. PIDX standards require that PIDX XML messages comply with RosettaNet version Delivery Header The delivery header identifies the sending and receiving trading partners, and the message instance information. This information is placed separately from the service header to allow access to the information by an intermediary when the service header is encrypted Service Header The service header identifies the PIP, the PIP instance, the activity, and the action to which this message belongs. It provides the process context for a message Payload The payload of a PIDX message includes the service content, and zero or more attachments. Only action messages can contain attachments. Document ID: V2.0 Page 20 of 51 Introduction to PIDX XML Transaction Standards v1_0

21 The payload is the content the service header describes and identifies. The Service Header format is fixed and independent of payload. The service content part of the payload depends on the PIP type and instance in action messages, and the processing status in a signal message. Any action message attachments depend on data requirements agreed to by the trading partners. 5 PIDX Partner Interface Processes A partner interface process (PIP) integrates the business processes of a buyer and a seller by exchanging messages containing common business documents. A PIP instance is a specific, unique PIP, such as sending a specific invoice for services. A PIP specification is a document describing the trading partner roles in the PIP, the activities each role performs, and the PIP message choreography. The PIP specification also defines the time, security, authentication, and performance constraints of the process activities. The PIP specification identifies the XML schemas that control the structure and content of the business documents exchanged. 5.1 PIP Model PIPs follow a specific choreography of action and signal message exchange. A PIP instance begins when a trading partner sends the first action message defined in a PIP specification and ends when all actions specified in the activity complete, or when one of the activities fails. RosettaNet describes the PIP model in three PIP views: The business operational view (BOV) The functional service view (FSV) The implementation framework view (IFV) The business operational view (BOV) and the functional service view (FSV) are based on ISO/IEC 14662:2004, the "Information Technologies Open-edi reference model." The RosettaNet specifications extended ISO/IEC 14662:2004 to include the implementation framework view (IFV), which describes TRP rules and constraints. The PIDX standards use the BOV and the FSV to describe PIPs, as defined in ISO/IEC 14662:2004. The information contained in RosettaNet s implementation framework view is contained in the BOV, FSV, and the trading partner implementation guides Business Operational View The business operational view (BOV) addresses the business processes and requirements trading partners use to execute a business transaction. These processes and requirements are specified in PIDX standards, trading partner implementation guides, government regulations (Sarbanes-Oxley, SEC regulations, etc.), and the business strategies, goals and policies of the trading partners implementing a PIP. The BOV defines the roles the trading partners play in the business transaction, and the activities each can perform. In the invoice PIP, the roles are "buyer" and "seller." Each role limits the activities trading partners can perform. The use case diagram in figure 6 illustrates the roles and the activities a buyer and a seller can execute when executing the Send Invoice PIP. Document ID: V2.0 Page 21 of 51 Introduction to PIDX XML Transaction Standards v1_0 Figure 6: Send Invoice PIP use case diagram

22 The use case shows that the seller role activities are limited to sending an invoice and validating the associated business signal sent by the buyer. The buyer role can validate the inbound action message and send a business signal acknowledging the seller s invoice. The use case in figure 6 describes what activities each role can execute in the PIP. The BOV also describes how each role will execute the activities: the processes each role executes to satisfy the use case requirements. The activity diagram in figure 7 shows how each role executes the PIP activities. Figure 7: Send Invoice PIP activity diagram Functional Service View The functional service view (FSV) is a technical view of the business transaction; it describes how business transactions defined in the business operational view (BOV) are executed by the trading partners information systems. The FSV describes the networks, endpoints, methods, and systems that will request and provide the business services described in the BOV. The FSV is governed by standards, specifications, and protocols including networking protocols (TCP/IP), communication protocols (HTTP/S), encryption standards (SSL), and others. The FSV defines the public interfaces each trading partner exposes to accomplish the integration goals defined in the business operational view (BOV). The BOV and the FSV are interdependent; the BOV depends on the FSV to implement the specifications and standards described in the BOV. The sequence diagram in figure 8 shows the message choreography, communications protocol, encryption standard, and the trading partner interfaces (URL) exposed in processes to implement the requirements shown in the Send Invoice PIP in figure 6. Document ID: V2.0 Page 22 of 51 Introduction to PIDX XML Transaction Standards v1_0

23 Figure 8: Send Invoice PIP functional service view sequence diagram The FSV also defines the message exchange controls for each of the actions and signals involved in the dialog. For actions, this includes specifying the time within which an Acknowledgment of Receipt signal should be sent; the time within which a response to the action should be sent (if applicable); whether authorization is required to perform the action; and whether a secure transport should be used to transmit the action to the recipient. Refer to the PIP specification for complete details of the FSV part of that PIP specification. 5.2 Defining a Partner Interface Process Before a buyer and a seller can execute a partner interface process (PIP) instance, they must describe the PIP sufficiently so that the two trading partners can configure applications to conform to each trading partner s business rules. The description of the PIP to be executed is described in two primary documents: 1. A PIP specification 2. Trading partner implementation guidelines PIP Specifications A PIP specification describes a set of well-defined public processes trading partners use to execute a PIDX PIP instance. PIP specifications define the roles each trading partner plays in the process, the activities involved, and the type, content, and sequence of business messages exchanged by the trading partners while engaged in these activities. PIP specifications describe the business requirements and processes in the Business Operational View (BOV) section, and the technical processes and requirements necessary to implement the business processes and requirements in the Functional Service View (FSV) section. The current version of the PIDX XML Transaction Standards do not contain PIP specifications. However, PIDX has developed PIP specifications for the next version of the standards. Those PIP specifications are backwards-compatible and could be used to describe the PIP implemented by the trading partners. PIP specifications can be found on the PIDX web site ( Document ID: V2.0 Page 23 of 51 Introduction to PIDX XML Transaction Standards v1_0

24 5.2.2 Trading Partner Implementation Guidelines Trading partner guidelines describe the payload data requirements necessary to process the sending trading partner s message. Payload data requirements include the allowed values of information items in the XML document specified by the PIP specification, and any required attachments. PIDX defines specific schemas that contain mostly optional information items that can be included in the service content of a business action message. There are multiple versions of each PIDX PIP schema. The trading partners executing the PIP must agree on the version, and which of the optional data are required. The PIP schema and version are usually specified by the trading partner receiving the business action message in that role s implementation guide. 6 PIDX and RosettaNet Differences There are some differences between the RosettaNet and PIDX standards. While the differences are small, some can cause compatibility issues with commercial off the shelf (COTS) software. 6.1 XML Schemas vs. DTDs The RosettaNet standards use document type definitions (DTDs) to define and validate message headers and service content. The PIDX standard uses XML schemas to define and validate XML service content. 6.2 PIDX Partner Interface Processes (PIPs) The needs of the oil and gas industry are sufficiently different from the electronics and related industries that RosettaNet PIPs could not be used to meet oil and gas company requirements. Accordingly, the Complex Products and Services Task Group (Com.Pro.Serv) developed PIDX PIPs specifically to meet the needs of the oil and gas industry. The PIDX PIPs are documented in various schemas, the Standard Master, and in the Implementation Guidelines. Unlike the RosettaNet PIPs, the current transaction standards do not have separate PIP specifications for each PIP. Trading partners and third party technology providers facilitating implementations need to provide their own PIP guidelines for their internal needs. 6.3 Two-action PIPs A trading partner may define a complete process that includes more than one business message. For example, a supplier may configure internal systems to send an invoice action message to a customer, and expect an invoice response action message from the customer. If the customer does not send an invoice response action message, the supplier s system will throw an exception that could cause manual intervention. RosettaNet allows multiple PIPs to be combined into a single PIP, such as the invoice/invoice response. However, the PIDX standards explicitly allow only single-action PIPs. While a trading partner can still define a process as including two messages, i.e. invoice/invoice response, a customer whose system sends Document ID: V2.0 Page 24 of 51 Introduction to PIDX XML Transaction Standards v1_0

25 an invoice response to a supplier, and includes a reference to the original invoice in the invoice response service header, is not sending a PIDX-compliant message. 6.4 PIDX Dictionaries The RosettaNet standards provide a dictionary framework to define the information items that can be used in a RosettaNet message. The framework includes technical and business data dictionaries, and message guidelines that provide necessary clarification and governance to the data defined in RosettaNet DTDs. The PIDX dictionary framework consists of the following: o PIDX XML Petroleum Industry Data Dictionary (PIDD) o Preamble Message Guideline o Delivery Header Message Guideline o Service Content Message Guideline PIDX uses schemas to define the content of action message service content, so message guidelines are not necessary. The PIDX service content information items are defined in the PIDX implementation guideline (IG) and their usage is constrained in individual message schemas, and the PIDX library schema. 6.5 Authorization RosettaNet states the authorization requirement should be specified in the PIP specification, and that trading partners must agree to authorization requirements in advance of executing a PIP. If the PIP and trading partners require authorization, RosettaNet requires that each message exchanged must be digitally signed. The PIDX standards state that trading partners must agree prior to initiating a messaging process that an initiating partner has the rights to send a specific business message and a receiving partner has the rights to act on that message. The standards do not require the sending trading partner to digitally sign the message. 6.6 Authentication The RosettaNet PIP specifications specify the authentication requirement of the PIP messages. If a PIP specification requires the message to be authenticated, then the message is authenticated by requiring the sender of the message to digitally sign the message. In RNIF 2.0, a RosettaNet Business Message is digitally signed following the S/MIME IETF (RFC 2311) specification. The PIDX standards require authentication, but the standards do not require the sender to digitally sign the message: PIDX requires used of an independently managed or independently verifiable means of message authentication for the exchange of business transaction for complex products and services. (PIDX XML Standards Master V.10, section Authentication) Document ID: V2.0 Page 25 of 51 Introduction to PIDX XML Transaction Standards v1_0

26 The PIDX standards allow digitally signed messages to authenticate trading partners. However, the majority of the companies in the oil and gas industry use Secure Socket Layer (SSL) protocol and firewall rulesets to verify who is sending and receiving PIDX messages. 6.7 Non-repudiation The RosettaNet Standards are straightforward on how and when to implement non-repudiation: The requirement is determined in each PIP specification. The PIDX standards are less clear, and different documents in the standard may conflict with others: The standards require reliable messaging and state that non-repudiation is a part of reliable messaging The standards require implementing systems to support non-repudiation methodologies such as digital signatures The standards recommend digital signatures, but they do not require them The standards do not address the inclusion of digests in business signals Non-repudiation of Origin and Content If a PIP requires non-repudiation, RosettaNet requires the trading partners executing the PIP must provide non-repudiation of origin and content as follows: 1. The message sender digitally signs the message 2. The message receiver stores the received signed message for a time agreed to by both trading partners PIDX requires message receivers to store messages for period agreed to by trading partners. PIDX recommends the use of digitally signatures, but does not require their use Non-repudiation of Receipt If a PIP requires non-repudiation, RosettaNet requires trading partners to provide non-repudiation of receipt as follows: 1. The receiver of the action message digitally signs the associated business signal message 2. The business signal must include an MD5 or SHA-1 digest of the action message 3. The receiver must store the action message for an agreed-upon period PIDX requires receivers to store messages for an agreed-upon period. The standards recommend the use of digital signatures, but they do not require their use. The PIDX Standards Master does not address the use of MD5 or SHA-1 digests in business signals, but the PIDX Implementation Guidelines includes, Define use of the message digest" as a decision that needs to be made in implementing a PIP. 7 Strengths of the PIDX XML Transaction Standards The PIDX Com.Pro.Serv (Complex Products and Services) Task Group chose to model the PIDX XML standards on the RosettaNet standards because the RosettaNet standards provide a robust and reliable messaging architecture for the exchange of XML documents. PIDX goals for the new XML standard include end-to-end security and integrity of both the messaging process and the transactional data. The Com.Pro.Serv. Task Group identified the following messaging requirements: Document ID: V2.0 Page 26 of 51 Introduction to PIDX XML Transaction Standards v1_0