UMIG 6.0. Position Paper B2B Communication. Versie v3.4 / Version v3.4 Voor implementatie / Pour implementation

Transcription

1 ATRIAS Market Processes UMIG 6.0 Position Paper B2B Communication Versie v3.4 / Version v3.4 Voor implementatie / Pour implementation Position paper MIG6 B2B communication v3.3.1

2 VERSIONS AND REVIEWS Version Management Reviews...4 INTRODUCTION Scope Context Drivers for the MIG6 B2B Communication Standard The MIG6 Interaction Model Business Process Integration...6 MIG6 INTERACTION MODEL REQUIREMENTS...7 B2B MARKET COMMUNICATION STANDARDS & PROTOCOLS Messaging Standards OSI Model Layer Protocols Application Layer Protocol Transfer of messages Application Layer Protocol - Transfer of reports Session Layer Protocol Transport / Network Layer Protocol Web Service Protocols for transfer of messages Overview of web service protocols used SOAP Protocol details Correlation Web Service Security Message Exchanges Synchronous service invocation Asynchronous message exchange Characteristics Web Service Description Web Service Reliability Idempotency Attachments Reliable polling Message Order Parallel receivers Multiple queues Parallel receive Parallel receive with message order ASYNCHRONOUS MESSAGE SERVICE SendMessage operation SendMessage Request Message SendMessage Response Message Duplicate detection GetMessage operation Position paper MIG6 B2B communication v /29

3 5.2.1 GetMessage Request Message GetMessage Response Message GetMessageCount operation GetMessageCount Request Message GetMessageCount Response Message IsAvailable operation IsAvailable Request Message IsAvailable Response Message Errors SYNCHRONOUS WEB SERVICE StartAccess WSDL TABELLEN & INDEXEN TABLES & INDEXES Table of Figures Position paper MIG6 B2B communication v /29

4 Versions and Reviews 1.1 Version Management Versie - Version Datum - Date Auteur Beschrijving - Description /07/2013 Initial draft /10/2013 Integration of comments received from the commercial market parties /11/2013 Integration of comments after BIM infosession of 19/11/ /06/ /09/2014 Integration of security specifications; small corrections and clarifications /06/2015 Synchronous operations /09/2015 Minor changes & additions + update of layout to be conform with other MIG6 documents /01/2016 Minor changes & additions 1.2 Reviews Reviewer Review version Review date General remarks Position paper MIG6 B2B communication v /29

5 Introduction 2.1 Scope Atrias will implement a Central Market System (CMS) for the Belgian energy market that facilitates the market processes by routing and managing MIG6 messages between the different actors in the market. This document describes the standards that will be used for the B2B Communication between commercial market parties and the CMS (see Figure 1, green arrows). Next to this B2B communication standard, we also distinguish a B2B communication between the TSOs and the CMS, and the DGOs and the CMS. Both are not in scope of this document, but are mentioned for the sake of completeness (see Figure 1). 2.2 Context Figure 1 - Overview interaction models for the B2B Communication Drivers for the MIG6 B2B Communication Standard One of the visions of MIG6 is to accelerate market processes towards a highly responsive model. The purpose is to reduce turnaround times by processing requests and events on-the-fly ( au fil de l eau ). By nature most of the MIG6 processes are not interactive (e.g. settlement processes, reports ) and will therefore be implemented in a decoupled, event-driven manner. However, due to the following reasons, an interactive model is desirable for a subset of the MIG6 processes: - Interactive pre-switches. - Possibility to integrate services in the call centers of the commercial market players. - Possibility to make information available on demand. MIG6 B2B communication choices are therefore determined by 1) the event driven nature of process interactions and 2) the interactive mode for a subset of these MIG6 processes The MIG6 Interaction Model The MIG6 interaction model supports three message exchange patterns (MEPs) from a business point-of-view: - MEP 1: Synchronous request / response: a commercial market party sends a business request to the CMS and a single business response will be returned. The requestor will actively wait for the reception of the business response before continuing with the next activity. - MEP2: Asynchronous request/response: a commercial market party sends a business request to the CMS and one or more business responses will be sent back. The requestor will not actively wait before continuing with the next activity. - MEP3: Asynchronous event: the CMS sends unsolicited business data to the market party, unrelated to an earlier request. These MEPs will be implemented with one or more HTTP Request/Response pairs which are detailed and illustrated in paragraph Position paper MIG6 B2B communication v /29

6 All market processes will be supported through asynchronous message exchanges. A subset of these market processes will also be supported with the synchronous request/response pattern. To be more precise, the message exchanges for the PreSwitchBasic and PreSwitchLight processes will also be supported through a synchronous patterns as well as the initiation of the StartAccess and MoveIn processes. Figure 2 - Synchronous interactions Business Process Integration The choreography of message exchanges between the market parties and the CMS is pre-defined. However, the actual business processes of the market parties and the business processes of the CMS are not linked and can evolve independently of each other. The collaborations between the market parties and the CMS have been formalized and agreed upon. They are described within the MIG6 sequence diagrams depicting the sequence of message exchanges between the objects needed to carry out the functionality of a scenario. The semantics and the format of the messages have also been formalized and agreed upon. Note that the CMS will never take the initiative to send a message to a commercial market party (irrespective of the message exchange pattern used, this also holds for e.g. loss messages and spontaneous rectifications ). The market parties will take the initiative to retrieve messages from the CMS. Position paper MIG6 B2B communication v /29

7 MIG6 Interaction Model Requirements Table 1 gives an overview of the requirements for the MIG6 interaction model in a formalized way. Requirement Category Reference Description General REQ-001 All B2B market communication must be based on electronic document exchange. REQ-002 REQ-003 REQ-004 REQ-005 Protocols that are used must be open: solutions requiring investments in proprietary software solutions must be avoided. Ad-hoc and high volume exchanges must be supported. Existing, mature and widespread technologies should be used where possible. Interactive message exchange must be supported. Messaging REQ-006 Uniform network and data transfer protocols must be used. REQ-007 REQ-008 REQ-009 The delivery of messages must be reliable (one-time and guaranteed delivery). Messages must be easily extensible. The delivery of messages with attachments must be supported. Security REQ-010 We need support for authentication, authorization, confidentiality and archiving. Interoperability REQ-011 We need a uniform message syntax and semantics for all market parties. Table 1 - MIG6 Interaction Model requirements Position paper MIG6 B2B communication v /29

8 B2B Market Communication Standards & Protocols Different standards and/or protocols have to be selected to fix the way the B2B Market Communication will be implemented between the CMS and the systems of the market players. We mainly need to discuss the following artifacts: - The messaging standards. - The protocol standards: the web service technology stack and the OSI Model Layer Protocol stack; the web service technology stack is built on top of the OSI Model Layer Protocol stack: Figure 3 - The web service technology stack built on top of the OSI Layer Protocol Stack 4.1 Messaging Standards For MIG6 data structures and message definitions, the ebix methodology was used to model the market information exchanges. The ebix methodology itself is based on UMM2 from UN/CEFACT as well as the use of Core components as specified by the ebxml Core Components Technical Specification (CCTS, ISO ). More background information is available in the document UMIG IM - XD BIM - Introduction and general principles. As part of MIG6, XML schemas (XSD) are available for all data structures that will be used in the message exchange between the market parties and the CMS. As a consequence, XML is used as markup language for all electronic data exchange within the MIG6 interaction model. 4.2 OSI Model Layer Protocols For the different abstraction layers of the OSI model (see Figure 3), the protocols that will be used in the MIG6 interaction model have to be selected: Figure 4 - The OSI-model layers (ISO/IEC ) Position paper MIG6 B2B communication v /29

9 4.2.1 Application Layer Protocol Transfer of messages - MEP 1 will be implemented with a single HTTP Request/Response pair (see Figure 4): Figure 5 - The implementation at application layer level of a synchronous request/response MEP 1 can be best illustrated for a pre-switch (see Figure 5): Figure 6 - MIG6 pre-switch sequence diagram and the corresponding mapping to a HTTP Request/Response pair - MEP 2 will be implemented with (at least) two HTTP Request/Response pairs (see Figure 6): 1) A first HTTP Request/Response pair will be used to send the business request. The first HTTP Response (technical ACK) will be used in order to accept or reject the business request on a technical level (authentication, technical validation). If accepted, the CMS will process the request in the background. 2) A second HTTP Request/Response pair will be used to retrieve the business response. The HTTP Request will be used to obtain the business response. The HTTP Response will be used to communicate the business response. Actually, any available response is retrieved which is then to be correlated with the corresponding request by the requestor. If multiple business responses will be generated by the CMS for a specific business request (which will typically be the case), an additional HTTP Request/Response pair will be needed for any new business response to be retrieved. As explained further down, the market party will continuously poll for available Business Responses (MEP2) and Business Events (MEP3). Figure 7 - The implementation at application layer level of an asynchronous request/response with two HTTP Request/Response pairs Request/Response pairs Position paper MIG6 B2B communication v /29

10 MEP 2 can be illustrated for a start access (see Figure 7): a first HTTP Request/Response pair will be used to send the request start access (ST-INT-1) and receive the accept (ST-IN-2) or reject (ST-INT-3). To retrieve the technical master data (ST-INT-5), an additional HTTP Request/Response pair is necessary to retrieve the technical master data. Note that in practice, multiple business responses will be available in response to a start access: next to technical master data, also metering data and a UTIL-FEE will be available. Both have to be retrieved using an additional HTTP Request/Response pair. Figure 8 - MIG6 switch sequence diagram and the corresponding mapping to a HTTP Request/Response pair - MEP3 with a single HTTP Request/Response pair Figure 9 - The implementation at application layer level of an unsolicited event with singe HTTP Request/Response pair Note again that the CMS will never take the initiative (as mentioned already in paragraph 2.2.3) to send a HTTP request to the system of a commercial market party. HTTP responses with code 5xx will only be returned by the CMS in case of technical faults. They will never be used to indicate business or application faults. Position paper MIG6 B2B communication v /29

11 4.2.2 Application Layer Protocol - Transfer of reports Note that MEP-2 might also be implemented differently: when large amounts of data have to be transferred from the CMS to the commercial market party, the request (or the subscription) will be handled using a HTTP Request/Response. However, the business response can be retrieved by the commercial market party by means of setting up an -connection with the CMS (see Figure 9): Figure 10 - The implementation at application layer level of an asynchronous request/response with a HTTP Request/Response pair and an SFTP connection Session Layer Protocol To meet REQ-010 for confidentiality, SSL and SSH are proposed. The combination of HTTP and SSL is better known as HTTPS, while the combination of file transfer over SSH is better known as SFTP. The use of HTTPS and SFTP ensures that all data is encrypted, without putting too much burden on the performance. Please note that message-level encryption will not be used in the MIG6 interaction model. Source IP filtering will be applied, requiring market parties to access the CMS from a limited list of fixed IP addresses. Mutual SSL authentication using client certificates will be implemented to authenticate market parties. For SFTP connections, the authentication will also be based on client certificates. Data integrity is provided by both SSL and SSH through the use of a keyed MAC algorithm. Protocol support: the CMS will provide support for TLS 1.2 onwards (SSL 3.0, TLS 1.0 and TLS 1.1 will not be supported). For SFTP, only SSH-2 will be supported. Certificates: Market parties have to provide a valid client certificate for SSL and SSH authentication. The only certificates that will be accepted are X.509v3 client certificates from a trusted public certification authority (CA). The minimum length of the RSA key pair should be 2048 bits and the key usage has to be Client Authentication Transport / Network Layer Protocol As all communication will take place over the public Internet, TCP and IP will be used as Transport and Network Layer Protocol. We do not need further specifications regarding the Data Link Layer and the Physical Layer as this is the choice of the ISP. Position paper MIG6 B2B communication v /29

12 4.3 Web Service Protocols for transfer of messages The use of HTTPS over TCP/IP in itself is not sufficient for the transfer of messages: not all requirements from chapter Error! eference source not found. are met. It has been decided to implement a Web Service protocol stack using the Web Service (WS) standards as basis to meet the B2B communication requirements. Main drivers for the use of Web Services are 1) support for both the synchronous interaction (MEP1) and asynchronous data exchange (MEP2 and MEP3) and 2) formal description of web service interfaces. Hence, all functionality available for the market will be offered via web-services. The Web Service protocol stack will be built on top of the HTTPS transport layer protocol. Only document-style web services will be used (wrapped style). The use of WS standards facilitates the use of the (XML) data structures explained in paragraph 4.1. The web services will be described with WSDL; the packaging will be done using the SOAP-protocol Overview of web service protocols used Figure 11 - The Web Service protocol stack layers Following table gives an overview of all selected WS standards that will be used. Category WS Standard(s) Description Requirement SOAP SOAP 1.1 The SOAP protocol is an XML-based protocol that is the foundation of the web service protocol stack we want to build. It defines the message format, message exchange patterns, protocol bindings, etc. It is ideally suited to transfer the MIG6 XML data structures and it is also widely adopted. The use of XML data structures and the SOAP protocol also permits easily extensions. Metadata Specifications WSDL 1.1 The Web Service Description Language (WSDL) will be used to describe the network services as a set of endpoints. Interoperability WS-I Basic Profile 1.2 This standard describes how SOAP 1.1 and WSDL 1.1 must interoperate. REQ-002, REQ-004, REQ-008 Cf. SOAP Cf. SOAP SOAP Protocol details Below follows a list with some constraints regarding the use of SOAP in the MIG6 interaction model (see Figure 11): - Every HTTP request contains one SOAP-envelope. - Every SOAP envelope contains one SOAP-body. - The SOAP header is not used. - The content of SOAP-bodies has been defined in MIG6 as messages (see MIG6 Business Requirements). - A SOAP-body contains one MIG6 message (request or response). - Every MIG6 message has a unique identifier, a GUID in the element Identification of the message header HeaderBEEnergyDocument - Some MIG6 message may contain multiple payloads as documented in the respective schemas, - Every payload contains a unique identifier which is a GUID. Position paper MIG6 B2B communication v /29

13 4.3.3 Correlation Figure 12 - A SOAP Message - wrapped style - with MIG6 containing singe payload The relation between messages e.g. request and response messages - will be based on the Identifiers provided in the payload(s) of the MIG6 messages (see Figure 12). These identifiers are GUID s. Every payload or transaction is assigned a GUID in the element Identification. If the first transaction (request of event) is the trigger for a number of subsequent and related exchanges a choreography - the identification of that trigger transaction should referenced in all subsequent message exchanges. A first response message will refer via the property ReferenceTransaction_Identification to the initial request. All other subsequent messages that are part of the choreography put this value in the property RelatedTransaction_Identification. Figure 13 - Identifiers in payload of MIG6 message used for correlation TODO!!! The document UMIG IM - XD BIM - Introduction and general principles elaborates further on the topic of transaction identification and correlation. Position paper MIG6 B2B communication v /29

14 4.4 Web Service Security Apart from the use of HTTPS to ensure confidentiality of the messages exchanged, the following general principles apply regarding authentication and authorization for the B2B market communication: - The CMS will only implement a coarse-grained authentication and authorization of the market parties having access to the CMS. Authentication will be based on client certificates. Only a market party as a whole will be authenticated and authorized. Fine-grained authentication and authorization such as end-users initiating a request via an internal application at the premises of the market party must be authenticated and authorized by the market party itself. - Once a trust relation has been established between a system from a market party and the CMS, all requests from the trusted system to the CMS will be processed. This implies that every request issued from the trusted system (that is controlled by an authorized market party) will be processed by the CMS. 4.5 Message Exchanges The overall principle of the CMS is to work with small chunks of information the MIG6 messages that are rapidly processed. In case of synchronous message exchanges (MEP1), the requests are processed immediately. For asynchronous message exchanges (MEP2 and MEP3), the incoming messages are processed in a matter of minutes and resulting return messages are rapidly available for retrieval. The continuous flow of messages is opposite to a slower, batch-oriented approach based on scheduling Synchronous service invocation Figure 14 - Synchronous service invocation With synchronous invocation of a service by a market party, the request message is immediately processed and the functional response returned in a matter of seconds. If the CMS or network connectivity is not available, the request cannot be made. If an error occurs during the call - timeout or connectivity issue the service call should be retried with exactly the same request message Asynchronous message exchange Figure 15 - Asynchronous message exchange With asynchronous communication, messages are first locally persisted and copied to the CMS for further processing. If the CMS or the network is temporarily unavailable, the messages are persisted and delivery is retried until successful. When sending a message asynchronously to the CMS, a technical ACK is immediately returned indicating the caller was authorized and the message passed technical validations. In case of a technical error, a SOAP Fault or HTTP error code is returned. Position paper MIG6 B2B communication v /29

15 Figure 16 - Asynchronous message exchange with technical ACK For messages from the CMS, the market party continuously polls with small intervals - for new messages. The messages from the CMS to the market party may either be - The result of an earlier message exchange from the market party to the CMS (MEP2), e.g. to initiate a process. - Or Messages from the CMS to the market party that are unsolicited (MEP3), triggered by an event in the CMS Characteristics Response time Response message Synchronous Near real-time (5-10 seconds average) Single response message Received in SOAP response Asynchronous HTTP connection Same as request Separate Technical ACK returned immediately One or more response messages Retrieved through polling #payloads in MIG6 message Exactly 1 Multiple payloads can be used Maximum MIG6 XML message size 5 MB 10 MB Position paper MIG6 B2B communication v /29

16 4.5.4 Web Service Description The CMS web services will be described using WSDL 1.1. The WSDL files will import the relevant MIG6 XML schema s used by the operations of the web service. Figure 17 - (preliminary) WSDL file for the StartAccess web service Notes: - The above example is preliminary, the final WSDL will definitely deviate from the above. - So called flat WSDL files containing all imported XML schemas will not be provided Splitting-up of large documents When large documents (> 10 MB) need to be exchanged, they will be split over multiple messages. Each individual message will be compliant with the message definition (XML schema). Messages will not be split at arbitrary locations. Support for large messages is modelled in the message definitions through support for sequences of messages. See also the document UMIG IM - XD BIM - Introduction and general principles Sending and receiving parties The sending and receiving parties are identified with a Global Location Number (GS1 GLN). A distinction is made between - the Sender and Recipient which are the functional sender and receiver of the Information - the Issuer and Addresse which are the parties at a technical level who actually transmit and receive the information. Position paper MIG6 B2B communication v /29

17 4.6 Web Service Reliability The Web Services and underlying HTTP protocol are unreliable in the sense that request or response messages may not arrive at their destination. The basic mechanisms to arrange for a reliable message exchange over HTTP are a retry mechanism and duplicate detection. Figure 18 - Reliable message exchange through retry and duplication detection The duplicate detection will be based on the unique identifier already present in the document header of the MIG6 messages (HeaderBEEnergyDocument) Idempotency In case a message is received twice, there are 2 possible behaviors: 1 st An error message is returned stating that the message was already received earlier 2 nd Return the response message already returned earlier The 2 nd option is called idempotency and will be implemented and supported by the CMS Attachments Some MIG6 messages are accompanied by a document. These documents are mostly binary (PDF, JPEG). These attachments will be base64 encoded (RFC 4648) and put inside the MIG6 XML message. This is accomplished by leveraging the BinaryObjectType (UN/CEFACT) core component. With the attachment being part of the message itself, the maximum message size includes the binary attachment and therefore implies that attachments - binary documents - are limited in size. For asynchronous interactions, the total message size must remain below 10 MB. The XML message plus base64 encoded attachments may never be bigger. For synchronous interactions, this maximum message size limit is set at 5 MB Reliable polling Figure 19 - BinaryObject containing base64 encoded PDF attachment As described earlier, the CMS will never establish connection towards market parties. To allow for reliable, asynchronous message exchange from the CMS to the market parties, the CMS will allow market parties to poll and retrieve messages. The mechanism to make this message retrieval reliable and robust is as follows: 1. A market party generates a unique ID (GUID) and persists this in a reliable store with status pending. 2. The market party invokes the GetMessage operation of the Message service, passing the unique ID as part of the request message. If this ID hasn t been used earlier to retrieve a message, the CMS Position paper MIG6 B2B communication v /29

18 a. Takes the next MIG6 message from the queue b. Archives the MIG6 message with the unique ID provided by the market party c. Returns the MIG6 message 3. The market party a. Receives the MIG6 message from the CMS and stores it reliably in its own message store b. Marks the self-generated unique ID as processed Figure 20 - Poll for new message from CMS If the CMS is called with the same unique ID a 2 nd time, the message returned earlier will be retrieved from the archive and returned again. This is in line with the idempotent behavior of the CMS. This allows a market party to retrieve the message again if there occurred a communication error or technical error. Figure 21 - Retry retrieval of message In case of a communication error, the market party should retry the retrieval of the message. No assumptions can be made if the CMS had already processed the request and had yes/no started returning the response message. It is suggested that the market party implements this message retrieval mechanism as 2 separate technical transactions: 1. Generate and store a new unique ID in a reliable manner; after having checked that no previously generated ID (with status pending) is still present, 2. Store the newly retrieved message in a local message store for further processing and mark the unique ID as processed. In case of a communication or technical error, it may be unknown if the CMS returned a message yes or no. To avoid any uncertainty, the GetMessage call should be repeated with the same unique ID until successful. A unique message ID can be marked as processed once an actual message is received (or the CMS returns empty message without payload - to indicate that no data is available). This implies that when re-starting its polling, the market party should first check if there remain message ID s with status pending. If yes, these message ID s with status pending - should be used first before starting to poll with newly generated message ID s. Please note that the unique ID used to retrieve messages is completely unrelated to any of the Identifiers within the MIG6 message itself. 4.7 Message Order In some MIG6 market domains - Structure and Measure - the order in which messages are processed is relevant Parallel receivers If a market party retrieves messages from the same queue with multiple receivers or threads, not all messages will arrive in the same order as they were sent. Position paper MIG6 B2B communication v /29

19 Figure 22 - Parallel consumers causing messages to be out-of-order As related messages may be fired rapidly after each other by the CMS, the likelihood of messages arriving out-of-sequence increases. The market parties may implement their own solution(s) for the proper handling of messages that arrive out-ofsequence. Alternatively, the CMS offers the option to receive messages on a technical level in the right sequence. The market party should use a single thread to retrieve the messages Multiple queues The CMS will spread the different message flows over multiple queues. This allows data from different market domains to be received with different delays. The messages in the Structure queue may be received faster by polling the queue with a higher frequency. Whereas messages from e.g. the Bill queue could be retrieved with a lower priority. Multiple queues allow multiple receivers to each retrieve messages from a single queue thereby retaining message order for that queue and market domain. Figure 23 - Queues per triggering market domain The queue a message is put on is determined by the market domain of the triggering process that generated the message. Depending on the business process, MIG6 messages of the same type may end up on different queues. Market Domain Structure, Measure, Rectification Structure Measure, not for billing Measure Billing Meter Reads Sample Message Types (non-exhaustive) ResponseStartAccess, TechnicalMasterData, MeterReadNonContinuousForBilling, IndividualGridFeeInvoice, ResponseMoveIn, ReconciliationMeteringVolumesByServiceDeliveryPoints, MeterReadContinuous, MeterReadForInformation, MeterReadNonContinuousForBilling, AggregatedMeteredDataForBilling, IndividualGridFeeInvoice, Position paper MIG6 B2B communication v /29

20 Market Domain Sample Message Types (non-exhaustive) ReconciliationMeteringVolumesByServiceDeliveryPoints, IndividualTransactionCostInvoicingInvoice, Bill AggregatedGridFee, IndividualTransactionCostInvoicingInvoice, Allocate PortfolioForBalanceResponsibleParty ProvisionalAllocationResultsForBalanceResponsibleParty AllocationResultsForBalanceResponsibleParty, Reconciliate ReconciliationResultsForBalanceResponsibleParty Messages for each of these queues are retrieved independently of one another. Figure 24 - Consumer(s) per queue For queues where message order is relevant, the market party should use a single thread or receiver Parallel receive For market domains where message sequence is not relevant or is implicit through big time differences between messages, messages may be retrieved in parallel with multiple threads. Figure 25 - Parallel consumers for single queue Position paper MIG6 B2B communication v /29

21 4.7.4 Parallel receive with message order If a market party wishes to retain message order but a single consumer would not provide sufficient throughput, the CMS can configure multiple queues for a single market party and market domain. The CMS arranges for the messages related to the same headpoint to be put on the same queue. Figure 26 - Parallel receive from 2 queues for single Market Domain Notes: Actually the modulo of the hash of the EAN will be used, to assure good distribution of messages. The number of queues supported is 1, 2, 4 or 8. The number is configurable, but fixed during runtime. Position paper MIG6 B2B communication v /29

22 Asynchronous Message Service For the asynchronous message exchange, the CMS will offer the Message web service. The SendMessage operation will be used to asynchronously send messages to the CMS. The GetMessage operation will used to continuously poll and retrieve messages from the queues on CMS side. Important note: the samples XML messages are not final and may still evolve before actual implementation 5.1 SendMessage operation SendMessage Request Message A market party that wishes to send a MIG6 message to the CMS invokes the SendMessage operation. <?xml version="1.0" encoding="utf-8"?> <soapenv:envelope xmlns:soapenv=" <soapenv:header/> <mh:sendmessage xmlns:mh=" <mh:payload> <! - MIG6 Message --> </mh:payload> </mh:sendmessage> </soapenv:body> </soapenv:envelope> The CMS will check the incoming message before accepting the message. The CMS will validate the incoming MIG6 message against its schema and execute extra technical validations. If an incoming MIG6 message contains multiple payloads, all of the payloads must pass the validation and technical checks. If one of the payloads is not compliant, the entire message is rejected. The CMS will for a single MIG6 message - never accept some payloads and reject others SendMessage Response Message When the message is successfully received, validated and persisted, the CMS will return a technical acknowledgement. <?xml version="1.0" encoding="utf-8"?> <soapenv:envelope xmlns:soapenv=" <soapenv:header/> <mh:sendmessageresponse xmlns:mh=" > <mh:messageid>9da50950-f30d-11e3-ac c9a66</mh:messageid> </mh:sendmessageresponse> </soapenv:envelope> Position paper MIG6 B2B communication v /29

23 5.1.3 Duplicate detection As explained earlier, a market party may re-submit a MIG6 message because of an earlier communication problem. If the CMS had already accepted the message but the acknowledgement was lost, the market party will (and must) re-submit the message. The CMS will detect if the message was already received earlier based on the Identifier in the DocumentHeader of the MIG6 message. The CMS will simply return the response returned earlier: in case of SendMessage this is the acknowledgement. 5.2 GetMessage operation GetMessage Request Message A market party that wishes to retrieve a message will generate a unique ID (GUID) and pass this in the GetMessage request. <?xml version="1.0" encoding="utf-8"?> <soapenv:envelope xmlns:soapenv=" <soapenv:header/> <mh:getmessage xmlns:mh=" <mh:messageid>9da50950-f30d-11e3-ac c9a66</mh:messageid> <mh:queue>structure_1</mh:queue> </mh:getmessage> <soapenv:envelope> The CMS will check its message archive to see if the message with the specified unique ID was already returned earlier. If so, the CMS will return that same message again. If the unique ID is new on CMS side, the CMS will check if messages are present on the specified queue. If messages are available on the queue, the next message is retrieved from the queue, stored with its unique ID in a message archive and returned to the market party GetMessage Response Message When a message is available on the queue for the market party, the CMS will return the message in the SOAP response. <?xml version="1.0" encoding="utf-8"?> <soapenv:envelope xmlns:soapenv=" <soapenv:header/> <mh:getmessageresponse xmlns:mh=" <mh:queue>structure_1</mh:queue> <mh:messageid>9da50950-f30d-11e3-ac c9a66</mh:messageid> <mh:documenttype>getmovein</mh:documenttype> <mh:payload> <! - MIG6 Message -->... </mh:paylod> </mh:getmessageresponse> </soapenv:envelope> Position paper MIG6 B2B communication v /29

24 If no data is available on the queue, the CMS will return a GetMessageResponse message with an empty payload. In such case, the unique ID may be re-used to poll for a new message. The market party should also pause during a period of time (minimum 10s) before polling for new messages. 5.3 GetMessageCount operation The GetMessageCount operation returns the number of messages available on a particular queue GetMessageCount Request Message <?xml version="1.0" encoding="utf-8"?> <soapenv:envelope xmlns:soapenv=" <soapenv:header/> <mh:getmessagecount xmlns:mh=" <mh:queue>structure</mh:queue> </mh:getmessagecount> <soapenv:envelope> GetMessageCount Response Message When a message is available on the queue for the market party, the CMS will return the message in the SOAP response. <?xml version="1.0" encoding="utf-8"?> <soapenv:envelope xmlns:soapenv=" <soapenv:header/> <mh:getmessagecountresponse xmlns:mh=" <mh:queue>structure²</mh:queue> <mh:count>15</mh:count> </mh:getmessagecountresponse> </soapenv:envelope> 5.4 IsAvailable operation The IsAvailable operation is used to check the availability of the endpoint. The operation may be used to continuously monitor the availability of the CMS. The IsAvailable operation can be used to e.g. implement the Circuit-Breaker pattern, whereby the SendMessage and GetMessage operations are only invoked if the CMS is available IsAvailable Request Message <?xml version="1.0" encoding="utf-8"?> <soapenv:envelope xmlns:soapenv=" <soapenv:header/> <mh:isavailable xmlns:mh=" Position paper MIG6 B2B communication v /29

25 </mh:isavailable> <soapenv:envelope> IsAvailable Response Message When a message is available on the queue for the market party, the CMS will return the message in the SOAP response. <?xml version="1.0" encoding="utf-8"?> <soapenv:envelope xmlns:soapenv=" <soapenv:header/> <mh:isavailableresponse xmlns:mh=" <mh:status>available</mh:status> </mh:isavailableresponse> </soapenv:envelope> Position paper MIG6 B2B communication v /29

26 5.5 Errors In case of lower level technical errors, the CMS will return an HTTP status code. If the authentication of the market party - based on the identity in its client certificate fails, the HTTP error code 403 will be returned. An HTTP error code 403 is also returned if the market party is not authorized (or certified) to call the Message Service. 401 Security Access Denied in case of issues obtaining the users identity. 403 Security Problem establishing SSL channel with client certificate 404 System Requested resource not found (e.g. incorrect SOAP address) 413 System Content length too large 500 System In case SOAP Fault or unidentified errors The errors below will be returned as a SOAP Fault message (with HTTP status code 500). System System System Security Syntax Syntax Syntax Syntax General Failure System configuration error Back-end timeout Requestor not authorized Technical message validation failed Payload (MIG6 message) not recognized (SendMessage operation only) Syntax validation failed for Business Message in Payload (SendMessage operation only) Unknown queue (GetMessage operation only) As mentioned earlier, the CMS will validate each payload of a message arriving via the SendMessage operation. If one of the payloads is not compliant, the entire message is rejected. The CMS will for a single MIG6 message - never accept some payloads and reject others. Position paper MIG6 B2B communication v /29

27 Synchronous Web Service To give an impression on how the synchronous web services supported by the CMS will look like, a preliminary version of the WSDL file of the StartAccess web service is provided below. 6.1 StartAccess WSDL <wsdl:definitions xmlns:soap=" xmlns:ws=" xmlns:wsdl= xmlns:xsd=" xmlns:msg=" name="requeststartaccess" targetnamespace=" <wsdl:types> <xsd:schema targetnamespace= elementformdefault="qualified" attributeformdefault="unqualified" xmlns:xsd1="un:unece:260:data:eem-requeststartaccess" xmlns:xsd2="un:unece:260:data:eem-responserequestnetuserinteraction"> <xsd:import namespace="un:unece:260:data:eem-requeststartaccess" schemalocation="../schemas/document/requeststartaccess/ebix_requeststartaccess_2016pa.xsd"/> <xsd:import namespace="un:unece:260:data:eem-responserequestnetuserinteraction" schemalocation="../schemas/document/responserequestnetuserinteraction/ebix_responserequestnetuserinteraction_2016pa.xsd"/> <xsd:element name="startaccess"> <xsd:complextype> <xsd:sequence> <xsd:element ref="xsd1:requeststartaccess"/> </xsd:sequence> </xsd:complextype> </xsd:element> <xsd:element name="startaccessresponse"> <xsd:complextype> <xsd:sequence> <xsd:element ref="xsd2:responserequestnetuserinteraction"/> </xsd:sequence> </xsd:complextype> </xsd:element> </xsd:schema> </wsdl:types> <wsdl:message name="startaccessrequest"> <wsdl:part name="req" element="msg:startaccess"/> </wsdl:message> <wsdl:message name="startaccessresponse"> <wsdl:part name="rsp" element="msg:startaccessresponse"/> </wsdl:message> <wsdl:porttype name="startaccess"> <wsdl:operation name="requeststartaccess"> <wsdl:input name="requeststartaccess" message="ws:startaccessrequest"/> <wsdl:output name="responsestartaccess" message="ws:startaccessresponse"/> </wsdl:operation> Position paper MIG6 B2B communication v /29

28 </wsdl:porttype> <wsdl:binding name="startaccesssoap" type="ws:startaccess"> <soap:binding style="document" transport=" <wsdl:operation name="requeststartaccess"> <soap:operation soapaction=" <wsdl:input name="requeststartaccess"> <soap:body use="literal"/> </wsdl:input> <wsdl:output name="responsestartaccess"> <soap:body use="literal"/> </wsdl:output> </wsdl:operation> </wsdl:binding> <wsdl:service name="requeststartaccess"> <wsdl:port name="tns:startaccesssoap" binding="ws:startaccesssoap"> <soap:address location=" </wsdl:port> </wsdl:service> </wsdl:definitions> Position paper MIG6 B2B communication v /29

29 Tables & Indexes 7.1 Table of Figures Figure 1 - Overview interaction models for the B2B Communication...5 Figure 2 - Synchronous interactions...6 Figure 3 - The web service technology stack built on top of the OSI Layer Protocol Stack...8 Figure 4 - The OSI-model layers (ISO/IEC )...8 Figure 5 - The implementation at application layer level of a synchronous request/response...9 Figure 6 - MIG6 pre-switch sequence diagram and the corresponding mapping to a HTTP Request/Response pair...9 Figure 7 - The implementation at application layer level of an asynchronous request/response with two HTTP Request/Response pairs...9 Figure 8 - MIG6 switch sequence diagram and the corresponding mapping to a HTTP Request/Response pair Figure 9 - The implementation at application layer level of an unsolicited event with singe HTTP Request/Response pair Figure 10 - The implementation at application layer level of an asynchronous request/response with a HTTP Request/Response pair and an SFTP connection Figure 11 - The Web Service protocol stack layers Figure 12 - A SOAP Message - wrapped style - with MIG6 containing singe payload Figure 13 - Identifiers in payload of MIG6 message used for correlation Figure 14 - Synchronous service invocation Figure 15 - Asynchronous message exchange Figure 16 - Asynchronous message exchange with technical ACK Figure 17 - (preliminary) WSDL file for the StartAccess web service Figure 18 - Reliable message exchange through retry and duplication detection Figure 19 - BinaryObject containing base64 encoded PDF attachment Figure 20 - Poll for new message from CMS Figure 21 - Retry retrieval of message Figure 22 - Parallel consumers causing messages to be out-of-order Figure 23 - Queues per triggering market domain Figure 24 - Consumer(s) per queue Figure 25 - Parallel consumers for single queue Figure 26 - Parallel receive from 2 queues for single Market Domain Position paper MIG6 B2B communication v /29