V 1.2 M.J.R. Hofland 1/28/2013
Referenced documents Name Owned by Date Version Status Messaging and Queueing Implementation Guidelines Jörgen Einarsson 1-11- 2002 2.0 How to implement HIT Rail Web Services Hitrail 28-01- 2013 1.0 UIC 918-1 Electronic reservation of seats/berths and electronic production of travel documents - Exchange of messages International Union of Railways October 2010 4.6 V1.2 January 28, 2012 2
Contents Referenced documents...2 Contents...3 1. Introduction...5 1.1. Queue Manager...5 1.2. Queues...5 1.3. Messaging and Queueing Interface...5 1.4. Triggering...5 1.5. Channels...5 1.6. Transmit Queues...6 2. Message...7 2.1. Correlation ID...7 2.2. Reply to queue...7 2.3. Time to live...7 3. Queue Manager...8 3.1. Naming Conventions...8 3.2. Port Number...8 4. Queue Configuration...9 4.1. Local Queues...9 4.2. Transmit Queues...9 4.3. Remote Queues... 10 5. Channels... 11 5.1. Naming Conventions... 11 5.2. Receiver Channel... 11 5.3. Sender Channel... 11 6. Requesting configuration on HEROS MQ server... 12 7. Messaging and Queueing Interface... 13 Appendix A Sequence Diagrams... 14 Normal Operation... 14 Connection error to Hitrail MQ server... 15 Detection error or XML validation error... 16 Translation error... 17 V1.2 January 28, 2012 3
Remote connection error... 18 Remote timeout error MQ to WS... 19 Response translation error... 20 Reply message is never sent by receiver... 21 V1.2 January 28, 2012 4
1. Introduction The basic idea for MQ is to keep the application separated from the communication and thus let the application do what it is best at, handling the data. MQ s promise to the application is that the data always will reach its destination. The following descriptions of MQ elements are taken from Messaging and Queueing Implementation Guidelines as they are still relevant. 1.1. Queue Manager The base in MQ is the Queue Manager (QM), the manager of all queues and connections. MQ handles the configuration and management of queues and channels. It gives the API (MQI) through which an application can access a queue. It can also start an application when a message has arrived on a queue. This function is called triggering. Normally, there is one QM on one host, but there is no limitation. You might as well have how many QMs you like on one host. 1.2. Queues In QM you define the queues. The queues can be local (on this QM) or remote (on another QM). You can also define alias queues for local or remote queues. A Local Queue is a queue from which an application can read a message. A Remote Queue is a definition of a queue that exists on another Queue manager. An application can put a message on this queue. When a message is put on the queue, Queue Manager takes care of the message and transmits it to its destination. An Alias Queue is a name of another queue, which can be local or remote. It can be used for various reasons, i.e. when you make a change at the receiving side and don t want to change the sending side. Examples how Alias Queues can be used is mentioned in Queue Configuration. 1.3. Messaging and Queueing Interface To read or put a message on a queue, an application accesses the Queue Manager and the queue over an API, MQI (Messaging and Queueing Interface). It is always the application that accesses the queue. 1.4. Triggering When a message is received by a receiving queue the Queue Manager can start an application that reads the queue. To do this, the local queue has to be configured as a triggered queue and know the application to be started. Only one application can be started to read from one queue. Triggering is not in scope of this document. 1.5. Channels To establish a communication between two Queue Managers you create channels. One channel for the sending to the other QM and one used for receiving from the V1.2 January 28, 2012 5
other QM. The channel is handled by a Message Channel Agent (MCA). The MCA establish the channels to other QM, and restarts the channel if necessary. The MCA handles all messages to queues on other QM. A channel can be configured to be established all the time or it can be configured to be established only when a message is to be sent. In an environment where you handle real-time application, the channel is configured to be established all the time. Carrier of the channels can be TCP/IP, SNA LU6.2 or NetBIOS. For HOSA TCP/IP is used as carrier. Channels are hidden to the application. 1.6. Transmit Queues Transmit Queues are defined for each channel for sending to a remote Queue Manager. If a message is destined to a queue on a remote Queue Manager, the local Queue Manager puts the message on the appropriate transmit queue. The MCA then reads the message and transports it to its destination. Transmit queues are hidden to the application. V1.2 January 28, 2012 6
2. Message The following description of an MQ message is taken from Messaging and Queueing Implementation Guidelines. A message consists of a header and application data. The header is called Message Descriptor (MQMD). The Message Descriptor contains information about the message which is used by both MQ and the receiving application. It is set by MQ and the sending application. MQ only reads the header. It never reads the application data. The receiving application can read both application data and header. The Message Descriptor is a set of attributes that describes the message and how the message is going to be treated by MQ and the receiving application. The following attributes are important for communication within HEROS: 2.1. Correlation ID The correlation id is used to connect the response message to the request message. 2.2. Reply to queue Name of the queue to where the reply is going to be sent. 2.3. Time to live This means that you set a lifetime for the message. If this time expires the message is not handed over to the receiving application. If the message is in the transmit queue when its lifetime is expired it is thrown away. The time to live is set by the sender. It can be set per message by the sending application. The option is set in the Message Descriptor. The expiry time is counted down in every queue (Transmit Queues and Destination Queue) the message passes. It is not dependant on that the Queue Managers have the same time. At a restart of Queue Manager, sending or receiving, the message is deleted if its lifetime is expired. V1.2 January 28, 2012 7
3. Queue Manager A queue manager manages all queues and channels. Creating a queue manager is the starting point for setting up MQ. 3.1. Naming Conventions In HOSA the following naming convention is used for the queue manager: HCCM H CC M used for = HERMES = country code. ISO-standard 3166. Alphabetic. = Serial number. Alphanumeric ( 0-9, A - Z). Even numbers = QM for production systems Odd numbers = QM for test systems. (A is even, B is odd, and so on. 0 is here counted as an even number and is production) Example: HGB0 = QM number 0 in Great Britain. 3.2. Port Number MQ uses port 1414 as standard port. Due to the use of proxy firewalls or that multiple Queue Managers running on the same host, other ports may be used. V1.2 January 28, 2012 8
4. Queue Configuration There are three types of queues that can be created: Local Queue, Transmit Queue and Remote Queue. Local queues are used to receive incoming messages. Remote queues are used to send messages to other queue managers. Transmit (or transmission) queues are used to store outgoing messages for a remote queue manager. In addition there is the Alias queue, which refers to another local or remote queue under a different name. 4.1. Local Queues When creating a local queue, the name of the queue should be according to the naming convention detailed below: HRRRRAASSMD H = HERMES RRRR = railway code. Decimal AA = UIC application number SS = UIC subapplication number M = mode (0 = Production, 1 to 5 = Test, 6 to 9 = reserved). D = direction (Q = queries, R = response) Example: H0070010Q = Receiving queue for queries from BR, application 01 in production. 4.2. Transmit Queues A transmit queue is created as a local queue. Set the property usage to transmission (default is normal) to change the local queue into a transmission queue. Name the transmission queue according to the following naming convention: Transmit queue for channel #1: RQM RQM = Receiving Queue Manager Example: HNL2 = Transmit Queue used for transmission to HEROS QM HNL2 on channel 01. Transmit queue for channel #2 (or higher): RQM.nn RQM = Receiving Queue Manager, nn=sequence number (from channel). Example: HNL2.02 = Transmit Queue used for transmission to HEROS QM HNL2 on channel 02. V1.2 January 28, 2012 9
4.3. Remote Queues When creating a remote queue, the name of the queue should be according to the naming convention detailed below: RQM.HRRRRAASSMD RQM = Receiving Queue Manager. HRRRRAASSMD = Queue on Receiving Queue Manager. Additional properties that need to be set on a remote queue are the name of the remote queue, the name of the remote queue manager and the name of the transmit queue that will be used for the remote queue manager. Example: HNL2.H0070010Q = the receiving queue for queries from BR to HEROS MQ server. H0070010Q = the remote queue HNL2= the remote queue manager HNL2= the transmit queue V1.2 January 28, 2012 10
5. Channels The communication between two Queue Managers is set up as a pair of channels. There is one channel established for each direction. Each side of a channel is handled by a Message Channel Agent. One channel consists of a Sender channel on the sending queue manager and a receiver channel on the receiving queue manager. When an application puts a message on a queue, MQ looks at the header. If the message is destined to a queue on another Queue Manager, the message is put on the transmit queue defined for this Queue Manager. The MCA then takes the message and sends it to the receiving MCA on the other Queue Manager. The receiving MCA then puts it on the destination queue. When the message is placed on its destination queue, it is removed from the transmit queue. 5.1. Naming Conventions QM1.QM2.nn Transmission from Queue Manager QM1 to Queue Manager QM2 nn = sequence number (01, 02, etc) Example: HGB0.HNL2.01 = Channel #01 for transmission from BR to HEROS MQ. QM2.QM1.nn Transmission from Queue Manager QM2 to Queue Manager QM1 Example: HNL2.HGB0.01 = Channel #01 for transmission from HEROS MQ to BR. 5.2. Receiver Channel Right click on the channels folder under your queue manager and choose New Receiver Channel.... Give the channel the name according to the described naming convention. 5.3. Sender Channel Create a Sender channel and name it according to naming convention. Configure the IP address of the remote queue manager under the property Connection name. Set the transmission queue to the transmission queue that was created for the remote queue manager. V1.2 January 28, 2012 11
6. Requesting configuration on HEROS MQ server In the new HEROS setup, MQ messages will be sent to a central MQ server. Each partner will have queues on this MQ server for receiving and sending messages. The middleware reads incoming messages from the queues and will take care of translating and delivering the message to the destination. This destination can be another (remote) queue on the central MQ server. This also happens for the response messages on the response queues from the recipient back to the original sender. The following information needs to be provided to Hitrail to be used to configure the central MQ server: Queue manager name IP address Port 1414) Local queue name HEROS will listen to Remote queue name server. HEROS will send Name of the partners queue manager IP address configured on the partners queue manager Port configured on the partners queue manager (default Name of the request queue on the central MQ server. incoming messages from this queue. Name of the response queue on the partners MQ messages destined for the partner to this queue. Channels will be created based on the supplied queue manager name. V1.2 January 28, 2012 12
7. Messaging and Queueing Interface An application accesses the queue on the queue manager via the MQI which is the interface to which the application sends its calls. The calls used by the application at MQI are: MQCONN Application connects to Queue Manager MQDISC Application disconnects from Queue Manager MQOPEN Application connects to queue MQCLOSE Application disconnects from queue MQPUT Application puts message on the queue MQGET Application reads messages from the queue MQINQ Application asks for the depth of the queue or which attributes are set. MQSET Application can set certain attributes for the queue. The standard procedure is to connect to the Queue Manager and open the queue for every message, or batch of messages, that is read from or put to the queue. After this the application closes the queue and disconnects from the Queue manager. More detailed information about using MQI can be found in Messaging and Queueing Implementation Guidelines. V1.2 January 28, 2012 13
Appendix A Sequence Diagrams This chapter describes H01 message flows with messages sent using MQ. It contains sequence diagrams for different H01 message flows and H01 error handling. An error message will be supplied in the format the sender expects, either XML or ASCII. A definition and detailed description of the XML error message Hermes Acknowledgement Message can be found in How to implement HIT Rail Web Services. ASCII error messages are H01 Negative Reply messages customized to contain the Hermes Acknowledgement error information. Information about H01 Negative Reply messages can be found in UIC 918-1 Electronic reservation of seats/berths and electronic production of travel documents - Exchange of messages. Normal Operation When the communication, between sender and receiver works as intended, then no error will of course occur. If the replying system responds within a set amount of time, the reply message can be handled within the same session that handled the request message. Within the middleware this is treated as a synchronous message flow. This is illustrated in the following sequence diagram using the MQ transport for sender and receiver.
Similar to when the receiver hosts a web service, the middleware treats this as a synchronous message flow. This situation should be handled by the sender as described in UIC 918-1 Electronic reservation of seats/berths and electronic production of travel documents - Exchange of messages under A.2.1 Normal operation. Connection error to Hitrail MQ server If the partners MQ server is unable to send the message to the central Hitrail MQ server, because of an MQ related error then the sending application will get an MQ error response. If the message could not be sent because there is no active channel between the sender s MQ server and the Hitrail MQ server then the message will stay on the sender s transmission queue. This situation occurs outside of the scope of HEROS. V1.2 January 28, 2012 15
Detection error or XML validation error After a message has been sent to the middleware it will acquire information by reading properties in the message (such as partner identifiers). If this fails because an identifier is not known or the message has an unknown format, a Hermes Acknowledgement error message will be returned according to the TAF TSI standard. If the message is in XML it will also be validated against the appropriate XSD. Any of the following error codes and message could be returned: 901 sender identifier unknown Connection is ok but problem with user configuration. The identifier in the message does not correspond with the HEROS configuration. 902 recipient identifier unknown Connection is ok but problem with partner settings. The identifier in the message does not correspond with the HEROS configuration. 903 configuration error Connection is ok but problem with partner settings. The client is not properly configured in HEROS. These errors require the client to check its HEROS configuration and verify that it corresponds with the messages sent. 904 xml validation error Connection and partner settings are good but problem with the message. The V1.2 January 28, 2012 16
request message did not pass the validation according to the XSD version configured in HEROS. This error requires the client to verify that the message sent is valid according to the XSD version that is configured under HEROS partner settings. Translation error If an error occurs in the translation step (transforming from one type or version of message to another) a Hermes Acknowledgement error message will be returned. Any of the following error codes and message could be returned: 905 mapping/transformer error Connection is ok but problem in translation. The request message could not be translated to the receiver s format or version. If this happens in production, a call should be made to investigate if the translating software needs a patch. V1.2 January 28, 2012 17
Remote connection error If an error occurs during the MQ communication step between the Hitrail MQ server and the receiving MQ server, then the message will remain on the transmission queue of the Hitrail MQ server until the connection is restored. In case of a channel failure, the error will occur on both MQ servers. When the connection is restored, all the pending messages on the transmission queues will be processed asynchronously. If the receiving system is connected using a hosting web service, then the middleware will retry sending the message a couple of times. If all attempts fail to send the message, a Hermes Acknowledgement error message will be returned. When the connection type used by the receiver is a web service, the following error code and message are returned to the sender: V1.2 January 28, 2012 18
906 remote connection error Connection and translation are ok, but could not connect to replying system after several tries. The sender cannot send any message to that particular receiving system until the connection has been restored by the responsible party. Sending messages to other systems is still possible. Remote timeout error MQ to WS The following situation describes when the request message has been correctly processed and sent to the receiver s web service. If the replying system does not respond before the middleware closes the session a Hermes Acknowledgement message will be send to the sender. The receiving system should therefore rollback the request when the session with HEROS closes before they are able to send the reply. 907 Remote WS connection timeout Connection and translation are ok, but could did not receive reply before the connection timed out. This situation should be handled by the sender as if the request message has never arrived at the replying system. Replying systems using a web service will rollback the request if they cannot send a reply. V1.2 January 28, 2012 19
Response translation error When the replying system sends the response message to the middleware it will need to be translated to the format the sender is expecting. If the translator cannot process the reply, the middleware will send a Hermes Acknowledgement message to the sender. This sequence diagram describes when the event happens in the synchronous flow and the receiver uses MQ. If the replying system is connected with a web service instead of MQ the same situation applies. V1.2 January 28, 2012 20
The following error code and message will be returned: 908 response mapping/transformer error Request has been sent to the partner and reply has been received, but there was a problem with the reply. This situation should be handled by the sender as described in UIC 918-1 Electronic reservation of seats/berths and electronic production of travel documents - Exchange of messages under A.2.3 - Defect before the reply is received from the replying system. Reply message is never sent by receiver This situation should be handled by the sender as described in UIC 918-1 Electronic reservation of seats/berths and electronic production of travel documents - Exchange of messages under A.2.3 - Defect before the reply is received from the replying system. V1.2 January 28, 2012 21