HP Service Virtualization Fixed Length Protocol Virtualization SV Training September 2014
Fixed Length Protocol Virtualization Technology Description Use Cases Supported Message Structures SV Service Description Format Virtual Service Summary Exercise 2
Technology Description TCP/IP layer is used directly Message is introduced by a field describing its length The same message type has usually the same length No delimiters are present Each field is described by offset and length pair Usually used as a legacy protocol or in embedded devices 3
Use Case 4
Use Case 5
Use Cases How to Recognize Fixed Length Message (While Viewing in Hex Editor) Unsupported characters are present in the beginning of the file (underlined) Lots of spaces (hex value 0x20) Lots of zeros (hex value 0x30) No visible data value separation Can read ASCII strings and numbers Free Hex Editor http://frhed.sourceforge.net 00000000 00 00 00 14 53 51 52 54 20 20 30 30 30 30 30 30...SQRT 000000 00000010 30 30 36 34 0064 6
Use Cases How to Recognize Fixed Length Message (From Provided Description) Offset and Length columns Format column Message Length field Alphanumeric and Numeric fields are mostly present Numeric fields are usually left zero padded to defined length Alphanumeric fields are usually right space padded to defined length Message Type field can be present All messages have a message length field on the same offset position Name Offset Length Type Description Value Length 0 4 Binary (network byte order) Message length Type 4 6 String Message type SQRT Number 10 10 Numeric Number 7
Supported Message Structures No occurring structures are supported (arrays of some type, defined field occurrence) Message can be shorter than defined, unspecified fields are set to not present value in SV Supported field data types: Format SV Format Note ASCII, Alphanumeric, String String Just ASCII chars are valid UTF-8, Unicode, Binary Any non-ascii charset Numeric Numeric Numbers in ASCII ( 000010 or -000020 ) Decimal Decimal Numbers in ASCII + decimal separator ( 0005.4 or -005.4 ) Fixed Decimal String Fixed decimal places (i.e. financial applications: 00100.00 ) Integer, Short (big-endian, network byte order) Integer, Short (little-endian, Intel x86) IntBE IntLE Supported byte length is 4 bytes or less. More info: http://en.wikipedia.org/wiki/endianness Binary, other non-specified Binary If some of parent formats is not working, use Binary instead. 8
Evaluation Direction SV Service Description Format (structure) Tab-separated table in *.txt file. Can be prepared in Excel and copy & pasted to *.txt file. Following table format is required (at any order), just bold fields are mandatory: Name Offset Length Type Description Default Value Attributes Length 0 4 IntBE Message Length 1 Type 4 4 String Message Type 2 Request Type Response Type REQ RESP Enumeration of matching message types MsgID 8 6 Numeric Message ID HasError 14 1 Numeric Has Error Flag 0 No Error 0 Error 1 Enumeration of possible values for the parent field Data 15 30 String Message Payload 9
SV Service Description Format (file name) Defined service description is used for request and response calls but if the file name matches one of the following patterns: Request File Name Pattern *_req.txt *_request.txt *_rq.txt *-req.txt *-request.txt *-rq.txt *.req.txt *.request.txt *.rq.txt Response File Name Pattern *_resp.txt *_response.txt *_rs.txt *-resp.txt *-response.txt *-rs.txt *.resp.txt *.response.txt *.rs.txt 10
Virtual Service Summary Multiple responses are supported TCP/IP protocol is only supported Multiple service description files can be used to create a virtual service Simulation from the scratch is not supported, learning is needed Just one virtual service can be deployed on a Fixed-Length TCP Agent Occurring structures are not supported Variable message structures (defined by a precedent field) are not supported Cobol strings are not supported (in a readable way) 11
Summary Introduced simple to use fixed-length protocol in SV No code changes needed for introducing customer-specific fixed-length protocol Easy maintenance of the protocol in customer environment Service description format can be used for further extensions (like Text Based protocol over TCP) 12
Exercise
Fixed Length Protocol Virtualization Exercise Use Case Application Analysis & Packet Dumps Service Description Creation Virtualization Troubleshooting Summary 14
Use Case Demo Application Simple calculator that supports SQRT and PLUS operations Responses are either RESULT or ERROR Demo Application Package Client Application (connecting to localhost:10000) Real Service (listening on localhost:10000) 15
Application Analysis & Packet Dumps Open *.pcap file in Wireshark Example *.pcap file Select the first TCP/IP packet with payload data Check the packet structure (the TCP part is the last part) Find the TCP segment data Check payload of the data Use the protocol recognition steps from the Lesson 5: Fixed Length Protocol Virtualization 16
Service Description Creation SQRT Operation Service Description File for SV Name Offset Length Type Description Default Value Data Attributes Length 0 4 IntBE Message length 1 Type 4 6 String Message type 2 SQRT Number 10 10 Numeric Number Example Excel with Service Descriptions 17
Service Description Creation PLUS Operation Service Description File for SV Name Offset Length Type Description Default Value Data Attributes Length 0 4 IntBE Message length 1 Type 4 6 String Message type 2 PLUS Base 10 2 Numeric Base Binary 2 Octal 8 Decimal 10 Hexadecimal 16 Number1 12 10 String Number 1 Number2 22 10 String Number 2 18
Service Description Creation RESULT Operation Service Description File for SV Name Offset Length Type Description Default Value Data Attributes Length 0 4 IntBE Message length 1 Type 4 6 String Message type 2 RESULT Base 10 2 Numeric Base Binary 2 Octal 8 Decimal 10 Hexadecimal 16 Result 12 10 String Result 19
Service Description Creation ERROR Operation Service Description File for SV Name Offset Length Type Description Default Value Data Attributes Length 0 4 IntBE Message length 1 Type 4 6 String Message type 2 ERROR ErrorCode 10 4 Numeric Error code ErrorMessage 14 1000 String Error message Output error_rs.txt file opened in Notepad++ 20
Virtualization Agent Configuration Virtual Service Port Number Real Service Hostname And Port Number (our example: localhost and 10000 ) 21
Virtualization Creating a Virtual Service & Learning 1. 2. 4. 3. 22 5.
Virtualization After Learning Protocol operations 23
Virtualization Data Model Refinement Message Length Recognition Note the length 24
Virtualization Data Model Refinement Fallback to the Error Message Type Create a new rule and set the message type to error_rs 25
Virtualization Data Model Refinement Default Rule Update Set all fields meaningfully, especially the length field must have the same value as the learned message When you hit the default rule, data model accuracy will be decreases 26
Troubleshooting The following namespaces can be set to TRACE log level to debug the protocol: HP.SV.MessageProcessing.GenericProtocol HP.SV.TcpAgent.FixedAndText HP.SV.MessageProcessing.ProtocolBuilder.GenericProtocolServiceDescriptionBuilder The protocol uses XML message format internally in SV The message logger also displays XML format, so don t be surprised Service description format is tab-separated, so check the format first when you get errors while creating a virtual service Fixed Length TCP Agent can be used just by one virtual service at the same time 27
Summary The same approach applicable also for Text Based protocol over TCP extension in HP SV Extensibility SDK No empty line should be present in a service description file Use Designer/Server logs for debugging Analyze messages in the message logger in Designer 28
Thank you!