Performance validation for the mobile core

Transcription

1 October 2015 Performance validation for the mobile core Are you ready for Terabits of Traffic?

2 EPC and virtualization, the impact on performance validation Performance validation for the mobile core 1 EPC and virtualization, the impact on performance validation Elastic and scalable performance validation for EPC GPP mobility performance validation Introducing TeraVM, mobile core performance validation solutions EPC performance validation, enabling terabits of traffic Defining a flexible test strategy for vepc Validation of call and traffic modelling at terabits per second Validation for RAN modelling Per bearer, per application performance validation What is the relevance of per bearer and real world users? Stateful bearers, real bearer traffic Voice Content Adding mixed traffic flows, support multiple bearers EPC performance validation strategy TeraVM EPC configuration options Signaling, Transaction and Capacity rate testing Performance validation for Voice over LTE (VoLTE/secure VoLTE) TeraVM voice application test scenarios Conclusion Appendices TeraVM available performance statistics TeraVM mobile core sample statistics TeraVM sample application statistics Dual Hosted VoIP Application Results Supported Voice Codecs Supported Video/Audio and Voice Codecs Table of Figures Figure 1 Platform validation scenario - opensource versus proprietary cloud managed platforms... 5 Figure 2 TeraVM EPC modular architecture emulate any of the core mobile nodes... 7 Figure 3 EPC performance validation using standard hardware... 9 Figure 4 TeraVM enables dynamic call and traffic modelling Figure 5 TeraVM emulating mobile handover in standard hardware Figure 6 TeraVM template configurations; subscriber profile and UE settings Figure 7 LTE handover templates; enable ease of configuration by simply selecting a profile Figure 8 TeraVM emulated RAN settings Figure 9 MME unique configuration settings per emulated MME Figure 10 TeraVM gateway emulation settings Figure 11 TeraVM emulating overload scenarios on EPC core Cobham - 2 -

3 EPC and virtualization, the impact on performance validation Performance validation for the mobile core 1 EPC and virtualization, the impact on performance validation In the age of virtual network functions (VNF), do we expect call modelling or traffic modelling to be accurate enough to enable selection of the correct components for the next generation of the mobile network core? Perhaps? However, the question many people are asking is - what impact do the founding pillars for Network Function Virtualization (NFV), as outlined in the ETSI ISG NFV whitepaper*, have on these theoretical modelling algorithms? For many carriers, NFV has the potential to enable greater agility such as ease of scale on their core platforms. Carriers may be looking to expand revenues sources by enabling a mobile virtual network operator (MVNO). Here the carrier may look to scale the mobile core by using with a virtual EPC (vepc) running on standard hardware. Traditional modelling techniques for capacity enables the carrier to make informed decisions on physical network infrastructure required to enable a network deployment at a given scale. However, for a vepc which can scale out/up on demand, this produces many unknowns, how will the core behave when running across a number of hosts, what happens if compute resources are not available, etc? vepc clearly offers many unique advantages, however carriers must have the confidence in the reliability and robustness of these components, ensuring the infrastructure and software components deliver the five 9 s of service reliability that carriers are custom to. To validate robustness and reliability requires a well defined test strategy, which is the purpose of this application note. However, before delving into a test strategy, it s worth considering what test tools/vendors are needed to facilitate performance validation for the elastic and virtual mobile network core. Carriers have made significant investment over the years, purchasing physical test appliances and many perceive that there is no need to change. At Cobham Wireless, we 2015 Cobham - 3 -

4 EPC and virtualization, the impact on performance validation know of carriers who started down this path, believing their physical test appliances which they connected to the datacentre infrastructure through physical interfaces were good enough. With our help they quickly realized that they were on the back foot in determining performance in the virtualization layer and when it came to validation of the VNF vendor, there was an even greater challenge. As the elastic core scaled, they had no way of following these virtual machine instances across the datacentre, or even dynamically scaling to ensure all new instances were included in the performance validation. With annual budgets already tight, the wrong test tool/vendor selection can result in projects been frozen for a year, which means at a corporate management level they are even further away from capitalizing on the benefits of NFV. At management level the question now becomes is it viable to continue to support modelling and proprietary hardware solutions for selection and performance validation of our mobile core, especially as it is no longer supporting our needs? What s the alternative? Clearly a new approach to validation of the mobile core is needed, one which enables elasticity and flexibility, facilitating ease of scale, ensuring a complete and accurate assessment of the new virtual mobile core. 1.1 Elastic and scalable performance validation for EPC As already highlighted, traditional modelling and use of proprietary hardware is no longer relevant especially when it comes to vepc performance validation. What s the alternative? The answer lies with the keywords of elastic and scale. TeraVM has from its inception adopted these core principles, underpinning the product architecture. Cobham Wireless TeraVM elastic test bed enables the user to scale as needed to meet the demand and scale of the deployment of the vepc Cobham - 4 -

5 EPC and virtualization, the impact on performance validation TeraVM is agnostic to the underlying hardware and virtualization layers, so it can be turned up in many places enabling carriers and vendors to performance test a number of deployment options. Many carriers start the performance validation cycle for vepc by evaluating and comparing the two deployment options: 1) proprietary technology e.g. VMware versus 2) opensource community equivalent e.g. OpenStack. Figure 1 Platform validation scenario - opensource versus proprietary cloud managed platforms In Figure 1 above, the vepc component will be deployed on both cloud managed platforms enabling carriers to determine the pros and cons of each option. From a test and measurement perspective this means the test solution selected must be deployable and managed on both platforms. This also means that TeraVM and vepc has support for multiple hypervisors i.e. KVM, ESXi, Hyper-v, etc Cobham - 5 -

6 EPC and virtualization, the impact on performance validation 1.2 3GPP mobility performance validation A fundamental challenge in validating the network core, is how to present and test the mobility aspects, this includes movement between cell locations or the user equipment (UE) handing over between radio access technologies (RAT), which may include fall back from 4G to 3G. As the vepc is deployed into datacentre like infrastructure, the challenge is how to represent multiple radio access technologies (RAT) in the datacentre. Physically, it s impossible to role in multiple radio heads and therefore necessary to emulate them. To enable accurate testing of call modelling, it will be necessary to represent the UE at scale but more importantly simulate the UE movement. 1.3 Introducing TeraVM, mobile core performance validation solutions TeraVM offers a complete end-to-end solution for physical and virtual EPC testing. TeraVM s modular architecture is made up of a number of virtual machines and is datacentre ready. Carriers can use standard hardware and/or orchestrate into their datacentres alongside the vepc. The modular architecture enables the necessary flexibility to wrap around the complete core or simply isolate a component of the EPC VNF deployment e.g. validate performance of the mobility management entity (MME) in standalone. A further advantage of the TeraVM approach, is that a single host can emulate the complete core functionality i.e. no need for multiples of dedicated proprietary units to fulfil various node functions. This enables the user to easily slot in any of the EPC nodes for wrap around testing; performance validate for bottlenecks in the VNF EPC deployment. Users of TeraVM can simply select the required interfaces to expose and load traffic on e.g MME expose S1-MME, S6a and S5, create load scenarios on interfaces Cobham - 6 -

7 EPC and virtualization, the impact on performance validation Figure 2 below highlights the core modules available in TeraVM for EPC performance testing. As the solution is built on standard hardware, users of TeraVM can easily scale to match the traffic model scenarios, up to a terabit of data plane traffic (S1-u). Figure 2 TeraVM EPC modular architecture emulate any of the core mobile nodes TeraVM enables end-end mobile core performance validation of: EPC control plane EPC user data plane TeraVM enables flexible control and data plan loading from a central pool of traffic, or an elastic test bed. A key reason to use TeraVM is the statefulness of the traffic which includes both the UE and applications Cobham - 7 -

8 EPC and virtualization, the impact on performance validation TeraVM supports stateful: UE procedures: ATTACH/DETACH, Authentication Mobility: Handover (3G->4G), Circuit Switch fallback (3G) o SRVCC (3GPP Rel. 8) o esrvcc (3GPP Rel. 10) o rsrvcc (3GPP Rel. 11) Application Traffic: Per UE, real applications of Voice, Video incl. MPEG-DASH and Data Together we can deliver the five 9 s of service reliability for vepc enabling continuity in service success for NFV deployments Cobham - 8 -

9 EPC and virtualization, the impact on performance validation 1.4 EPC performance validation, enabling terabits of traffic The purpose of the application note, is to outline a test strategy for EPC performance validation, enabling highly scaled load capability up to 1Tbps. One which is built on standard hardware, helping to keep costs minimal. In doing so, it will introduce a new concept of the elastic test bed, enabling ease of scale. Figure 3 EPC performance validation using standard hardware Figure 3 outlines the basic architecture of the test bed built on datacentre ready hardware. TeraVM is used to simulate the scenario of UEs attaching to the network and sending traffic on the S1-u interface, enabling up to 1 Terabit per second of user plan traffic. This application note will define the test strategy and will outline the relevance of per bearer per application performance validation Cobham - 9 -

10 Defining a flexible test strategy for vepc 2 Defining a flexible test strategy for vepc 2.1 Validation of call and traffic modelling at terabits per second Call modelling focuses on the UE capabilities, the ability to connect to the mobile network. In this use case TeraVM delivers a number of templated scenarios e.g. UE attach, authentication and detach after a period at the scale of millions of UEs. The dynamic capability of the TeraVM means the emulated load can then be dynamically switched from the S1 interface to S1-u modelling the traffic scenarios on the carrier network. Real world load scenarios require equivalent load capabilities at terabits per second on the S1-u interfaces. TeraVM running on standard hardware makes it extremely easy to scale in a cost efficient way, without breaking the bank. Figure 4 TeraVM enables dynamic call and traffic modelling 2015 Cobham

11 Defining a flexible test strategy for vepc 2.2 Validation for RAN modelling TeraVM also facilitates RAN simulation for enodebs registering, management and procedure calls (S1AP), enabling greater realism of the test cases. TeraVM supports S1AP E-RAB management procedures for setup, modify and releases. In addition TeraVM supports location reporting procedures. The TeraVM RAN simulation enables the core test case for mobility and handover scenarios. Figure 5 TeraVM emulating mobile handover in standard hardware 2015 Cobham

12 Per bearer, per application performance validation 3 Per bearer, per application performance validation 3.1 What is the relevance of per bearer and real world users? Per bearer in its purest form means no two bearers are the same, exactly how real world subscribers connect and communicate on the network today. Per bearer coupled with user application emulation enables testing with flows with precise details and properties i.e. device credentials IMSI, IMS SIP configurations and stateful media for Voice and/or Video over LTE. The true benefit to per bearer testing is the ability to provide performance metrics on each and every UE and the associated application flows such as SIP and RTP. The importance of a granular approach becomes evident for policy and charging, ensuring that each UE and activity are accurately recorded, ensuring subscribers are billed/managed correctly, while active on the network. To further enhance the test strategy, the test should easily scale to the correct number of unique subscribers. For example TeraVM supports up to 8,000,000 active bearers each with unique activities i.e. making calls, sending s or even watching video. Note: In reference to TeraVM application flows, each flow is a 5-tupple, in which all elements of the flow are configurable: 1. Source IP 2. Destination IP 3. Source Port 4. Destination Port 5. Protocol 2015 Cobham

13 Per bearer, per application performance validation 3.2 Stateful bearers, real bearer traffic In the real world, voice conversations are generally two way or bi-directional therefore it should be considered as a pre-requisite for the test strategy that the traffic flows are stateful and bi-directional i.e. understand and respond to individual and unique control and transport layer requests. The purpose of using stateful traffic flows in EPC test strategies include: 1) Real calls and codecs (NB and/or WB- AMR codecs). a. Using fixed, stateless, bit patterns offer no reflection of real network conditions, offering no perspective on user quality of experience 2) Connect to auxiliary services such as IMS 3) Determine the impact/accuracy that network management (e.g. PCEF) has on an individual endpoint s performance. 4) Percentage of overall network bandwidth given over to control signals versus goodput of actual application flows. 3.3 Voice Content It s vital to add real voice samples to each bearer, which may include encrypted RTP media. An ideal test will be to use different codecs and encryptions. This helps to determine network reliability but is also useful in testing SIP proxy or call handlers to block calls between endpoints which are incompatible. 3.4 Adding mixed traffic flows, support multiple bearers The mobile core will support multiple bearers. Performance testing must include a mix of traffic from a range of applications which includes the use of multiple APN assignments. In short, the greater the variety of subscriber activities the more realistic the testing scenarios become, offering the greatest amount of test coverage and efficiency in testing Cobham

14 EPC performance validation strategy 4 EPC performance validation strategy 4.1 TeraVM EPC configuration options TeraVM s integrated configuration and performance measurements facilitates users with flexible configuration options. TeraVM is template driven, enabling users to quickly select any number of scenarios for call and traffic modelling, from a centralized library of tests. Templates enable ease of use, providing a minimal touch configuration experience Subscriber settings Users can configure up to 8,000,000 active bearers with the ability to assign unique traffic profile templates e.g. Teenagers - lots of video activity, Commuters simulating movement, consuming lots of data bandwidth, Business mixed voice and data applications such as . Figure 6 TeraVM template configurations; subscriber profile and UE settings 2015 Cobham

15 EPC performance validation strategy In addition, users can assign unique UE credentials of IMSI, MISDN and unique APN. TeraVM supports multiple bearers per UE, each bearer supporting any mix of user applications. TeraVM simplifies user configurations via the use of templates. Figure 7 below is an example profile for commuters; simulating mobility. This results in all the UEs in the associated profile taking part in the simulated handover for Inter MME. Figure 7 LTE handover templates; enable ease of configuration by simply selecting a profile The emulated UE supports stateful network access stratum (NAS) procedures (Attach / Authenticate / Mobility / Integrity / Cipher). This provides for effective call modelling capacity validation strategy, in which the result is based on the number of NAS (signaling events) per busy hour Cobham

16 EPC performance validation strategy RAN configuration options TeraVM enables users to configure multiple RAN types: eutran, UTRAN supporting mobility test cases for enodeb inter-lte radio handovers and use cases for 4G to 3G fallback. TeraVM enables users to configure unique credentials of PLMN and Cell IDs for the RAN under emulation. Figure 8 TeraVM emulated RAN settings Note: TeraVM is 3GPP compliant, supporting the following mobility procedures: Handover (3G->4G) Circuit Switch fallback (3G) o SRVCC (3GPP Rel. 8) o esrvcc (3GPP Rel. 10) o rsrvcc (3GPP Rel. 11) 2015 Cobham

17 EPC performance validation strategy MME settings TeraVM supports emulation of multiple MME with unique configuration parameters. TeraVM MME is fully stateful and supports S1AP messaging up to 60,000 messages per second. The emulated MME complies with 3GPP RAN setup and UE handover procedures, which include tracking area updates per emulated UE. Figure 9 MME unique configuration settings per emulated MME 2015 Cobham

18 EPC performance validation strategy Gateways and Application server settings TeraVM emulates all the gateway functions of the EPC, which include emulation of IMS services. TeraVM stateful application emulation enables users to emulate their own application servers or indeed connect to a 3 rd party service for voice, video and data applications. The stateful gateways and interface means users can plug-in their own nodes and validate performance of the node which includes MME, HSS and PCRF. Figure 10 TeraVM gateway emulation settings 2015 Cobham

19 EPC performance validation strategy 4.2 Signaling, Transaction and Capacity rate testing The following sections summarizes the required test steps to enabling a complete validation strategy for EPC. EPC performance validation can be broken down into a number of core steps enabling an efficient testing procedure ensuring the widest coverage, this includes testing for signaling, transactions and capacity rates. TeraVM provides templates for each scenario Signaling validation procedures Management Procedures o S1 Setup Attach o EPS Attach o EPS/IMSI Combined Attach o GUTI Attach o Emergency Attach Tracking Area Updates o Inter-MME Tracking Area Update o Intra-MME Tracking Area Update Intra-LTE Handover o S1-based HO (MME unchanged) o S1-based HO (MME changed) Inter-RAT HO o Inter-RAT PS Handover E-UTRAN to UTRAN (SGW unchanged) o Inter-RAT PS Handover E-UTRAN to UTRAN (SGW changed) o Inter-RAT PS Handover UTRAN to E-UTRAN (SGW unchanged) o Inter-RAT PS Handover UTRAN to E-UTRAN (SGW changed) Bearer Modification o UE initiated bearer modification with bearer QoS Update Transaction rate validation Attach / Detach rate Bearer Activation/Deactivation rate Dedicated Bearer Activation/Deactivation rate Tracking Area Update Rate EPC capacity test procedures Maximum number of enb/henb per MME/SGW Maximum number of UE s per MME/SGW Maximum number of GTP tunnels per SGW Maximum number of supported APN s 2015 Cobham

20 EPC performance validation strategy 4.3 Performance validation for Voice over LTE (VoLTE/secure VoLTE) An EPC test strategy will include validation of both the UE and voice application UE Assessment: How fast can the UE attach and authenticate with the MME/HSS? Provisioning the bearer how fast can the EPC allocate multiple bearers? IMS services how quick is the SIP proxy/registration authentication process? Inbound / Outbound Connectivity Is the SIP proxy server routing configuration correct? Media Encryption Can calls be sustained with encrypted media flows (SRTP)? Call Media Quality How much latency is tolerable by the subscriber? The above is a sample of just a single UE s media session, in real world terms the process occurs over thousands of unique UEs. Therefore testing needs to be scalable, not forgetting that communication is a two way flow, with various handshaking and negotiations e.g. (SIP Invite/Bye, TCP window resizes, etc). Scalability Test Scenarios: Oversubscription of UEs Examine the fallout when all UEs attach at the same time, will the EPC scale out? Call throughput Determine the maximum concurrent calls possible, in a pure VoIP only situation. Re-examine when additional traffic types are added to the mix? Emergency Call handling Using the above examples, examine how 911 calls are handled Cobham

21 EPC performance validation strategy Figure 11 TeraVM emulating overload scenarios on EPC core 2015 Cobham

22 EPC performance validation strategy 4.4 TeraVM voice application test scenarios 1. SIP Registration Test functionality and performance timing. Connect and register with the SIP registration server with unique details and passwords. Determine maximum number of registrations possible on a per second basis. 2. Interact with 3 rd Party Servers Emulated endpoints are fully stateful, and can interact with a number of leading vendor s equipment e.g. connect to a IMS service, plus make RTP and SRTP calls. 3. Call Quality - Establish a baseline reference call for voice quality establish the most suitable codec and optimal buffer sizes for devices under varying networking conditions. 4. Bulk Call Capabilities Establish the maximum number of calls possible, use a mix of traffic flows with signaling only and signaling plus media. 5. Load Testing (up to terabits of S1-u load) - Test and measure voice quality under varying traffic conditions and network loads, on a per client / per flow basis. Mix several unique individual voice, multicast video and multiple data clients running service applications. Run true, stateful TCP based application flows along with voice flows with multiple codecs. Exchange real voice, access real multicast streams, documents, URLs and attachments in order to emulate realistic, per client traffic flows. 6. IPv6 transition testing Utilize unique layer 2/3 properties, concurrently examine performance of IPv4 versus IPv6 enabled voice applications. Examine performance of dual stack enabled UACs, investigate the ability to register/connect calls using IPv4 or IPv6. This provides a unique MAC and IP address per client. The flexible MAC address configuration and unique options assignment is key for validating security in many environments Cobham

23 EPC performance validation strategy 7. IMS enabled Multimedia calls Analyze the SIP registration performance. Examine RTP performance and the incoming media quality. 8. Secure calls through TLS/SSL Measure and compare performance of secure (TLS/SSL) and unsecure voice calls. Determine if by varying the codec type has any influence on performance in TLS/SSL sessions. 9. NAT boundary traversal - Measure the effects of NAT boundary traversal (RFC 3261) on call quality. 10. Disruptive flows (P2P, DDOS, IGMP floods, spam, and viruses) add to the existing test scenarios, test and verify any security and mitigation rules or functionality that may be available. 11. Network device QoS Settings - Run individual voice, multicast and application data on emulated hosts against external voice, multicast and application data servers. Test and verify appropriate QoS mechanisms to use at L2 and/or L3/4 to classify traffic into each service category. Assign VLAN priority (on single and tunneled QinQ) on emulated endpoints and DiffServ/TOS classification for each individual application/service Cobham

24 Conclusion 5 Conclusion The TeraVM mobile network core performance validation solution provides unprecedented scale when it comes to validation of mobile cores. TeraVM delivers the realism of modern mobile network subscribers, enabling validation with terabits of subscriber application traffic. TeraVM is a virtualized solution, based on a modular architecture, which provides numerous mobile test use cases, enabling validation for end-to-end mobile scenarios or is used to validate unique mobile core nodes i.e. in a wrap-around scenario. A key reason for choosing TeraVM is the realism of its UE and applications. TeraVM enables unique configurations per UE and enables users represent mobility of subscribers i.e. traversing a 4G network or falling back to 3G. TeraVM is fully compliant with 3GPP standards. The use of standard hardware makes it extremely cost efficient to scale to terabits of traffic for validation of the mobile core. TeraVM s elastic test bed offers unique capabilities in scaling the test bed and also in terms of managing test scenarios. TeraVM offers a centralized test library ensuring any knowledge learned is easily accessible by other users. TeraVM s templated approach to enabling subscriber, UE and traffic scenario profiles, makes it extremely easy for users to quickly represent unique subscribers and applications on a per bearer basis. Per bearer validation enables a highly accurate interpretation on how the mobile core will perform under varying models of call and traffic load scenarios. TeraVM is NFV ready, which means carriers and equipment vendors can deploy the validation solution alongside their vepc in the datacenter cabinets. TeraVM is the only solution to offer mobility test scenarios for EPC VNFs. Validation includes handover scenarios like 4G to 3G - all on standard hardware. TeraVM is the only solution which delivers proven scale of terabits of traffic for performance validation of mobile network cores, in a cost effective manner Cobham

25 Appendices TeraVM available performance statistics 6 Appendices TeraVM available performance statistics 6.1 TeraVM mobile core sample statistics Sessions Initiated Sessions Succeeded Sessions Failed Sessions Failed Due To Resource Exhaustion IPv4 Sessions Initiated IPv6 Sessions Initiated Rx Attach Request Tx Attach Accept Rx Attach Complete Tx Attach Reject Rx PDN Connectivity Request Tx Activate Default Bearer Request Rx Activate Default Bearer Accept Tx Information Request Rx Information Response The number of default and dedicated bearer sessions that have been initiated. The number of default and dedicated bearer sessions that succeeded. The number of default and dedicated bearer sessions that failed. Number of Attaches rejected because internal resources were exhausted. This indicates that the UE count may need to be increased or the Attach load is not balanced between CPUs. The number of IPv4 bearer sessions that have been initiated. The number of IPv6 bearer sessions that have been initiated. The number of Attach Requests received from the UE, as part of an attach procedure. The number of Attach Accept messages sent from the network to the UE. This message is sent to the UE to indicate that the corresponding attach request has been accepted. The number of Attach Complete messages received from the UE. This message is sent by the UE in response to an Attach Accept message. The number of Attach Reject messages that the network sent to the UE. This message is sent to the UE to indicate that the corresponding attach request has been rejected. The number of PDN Connectivity Request messages received from the UE. This message requests establishment of a PDN connection. The number of Activate Default EPS Bearer Context Request messages sent to the UE. This message requests activation of a default EPS bearer context. The number of Activate Default EPS Bearer Context Accept messages received from the UE. This message acknowledges activation of a default EPS bearer context. The number of Information Request messages sent to the UE. This message is sent to the UE to request ESM (EPS Session Management) information (protocol configuration options, APN, or both). The number of Information Response messages received from the UE. This message is sent by the UE to the network in response to an Information Request message; it provides the requested ESM (EPS Session Management) information Cobham

26 Appendices TeraVM available performance statistics Rx Detach Request (switch off) Rx Detach Request (normal) Tx Detach accept Tx Authentication Request Rx Authentication Response Rx Authentication Failure Tx Authentication Reject Tx Security Mode Command Rx Security Mode Complete Tx Identity Request Rx Identity Response Rx Security Mode Reject Tx Paging Rx Service Request The number of Detach Requests received from the UE, in which the Switch Off parameter indicates that the request is the result of a switch off situation. The number of Detach Requests received from the UE, in which the Switch Off parameter indicates that the request is the result of a normal detach (and not the result of a switch off situation). The number of Detach Accept messages sent to the UE received. This message is sent to the UE to indicate that the UE-originating detach procedure has been completed. The number of Authentication Request messages sent to the UE. This message initiates authentication of the UE identity. The network initiates the Authentication Procedure by sending an Authentication Request message to the UE and starting the T3460 timer. The Authentication Request message contains the parameters necessary to calculate the authentication response (refer to 3GPP TS for detailed information). The number of Authentication Response messages received from the UE. This message delivers a calculated authentication response to the network. The number of Authentication Reject messages received from the UE. (The MME sends an Authentication Request to the UE, and the UE responds with either an Authentication Response or a Reject message.) The number of Authentication Reject messages sent to the UE. The Authentication Reject message is sent by the network to the UE to indicate that the authentication procedure has failed and that the UE must abort all activities. The number of Security Mode Command messages sent to the UE. This command is used to establish NAS signaling security. The number of Security Mode Complete messages received from the UE in response to the Security Mode Command messages received. The number of Identity Request messages that the network sent to the UEs. This message is sent by the network to the UE to request that the UE provide the specified identity. The number of Identity Response messages that the network received from the UEs. This message is sent by the UE to the network in response to an Identity Request message; it provides the requested identity. The number of Security Mode Reject messages received from the UE. This message indicates that the corresponding Security Mode Command has been rejected. The number of Paging messages sent to the UEs. The number of Service Request messages received from the UEs. The service request message is sent by the UE to the 2015 Cobham

27 Appendices TeraVM available performance statistics network to request establishment of a NAS signaling connection and establishment of radio and S1 bearers. Tx Service Reject ESM Insufficient Resources ESM Unknown or Missing APN ESM Auth Failed ESM Request Rejected by Serving GW or PDN GW ESM Request Rejected, unspecified ESM Regular Deactivation ESM Network Failure ESM PDN type IPv4 only allowed ESM PDN type IPv6 only allowed ESM information not received EMM Unknown IMSI EMM IMEI not accepted EMM UE cannot be derived by network EMM Network Failure EMM ESM Failure EMM Congestion The number of Service Reject messages sent to the UEs. This ESM cause is used by the UE or by the network to indicate that the requested service cannot be provided due to insufficient resources. This ESM cause is used by the network to indicate that the access point name was not included or could not be resolved. This ESM cause is used by the network to indicate that the requested service was rejected by the external packet data network due to a failed user authentication. This ESM cause is used by the network to indicate that the requested service or operation or the request for a resource was rejected by the Serving GW or PDN GW. This ESM cause is used by the network or by the UE to indicate that the requested service or operation or the request for a resource was rejected due to unspecified reasons. This ESM cause is used to indicate a regular UE or network initiated release of EPS bearer resources. This ESM cause is used by the network to indicate that the requested service was rejected due to an error situation in the network. This ESM cause is used by the network to indicate that only PDN type IPv4 is allowed for the requested PDN connectivity. This ESM cause is used by the network to indicate that only PDN type IPv6 is allowed for the requested PDN connectivity. This ESM cause is used by the network to indicate that the PDN connectivity procedure was rejected due to the ESM information was not received. This EMM cause is sent to the UE if the UE is not known (registered) in the HSS". This cause is sent to the UE if the network does not accept an attach procedure for emergency bearer services using an IMEI. This EMM cause is sent to the UE when the network cannot derive the UE's identity from the GUTI/S-TMSI/P- TMSI and RAI e.g. no matching identity/context in the network or failure to validate the UE's identity due to integrity check failure of the received message. This EMM cause is sent to the UE if the MME cannot service an UE generated request because of PLMN failures. This EMM cause is sent to the UE when there is a failure in the ESM message contained in the EMM message. This EMM cause is sent to the UE because of congestion in the network (e.g. no channel, facility busy/congested etc.) Cobham

28 Appendices TeraVM available performance statistics EMM Message type not implemented Rx S1 Setup Request Rx S1 Setup Request Tx S1 Setup Response Tx S1 Setup Failure Rx Initial UE Msg Tx DL NAS Transport Rx UL NAS Transport The number of EMM messages that returned EMM cause code 97 (Message type non-existent or not implemented). The number of S1 Setup Request messages received from the enodeb. This message is sent by the enodeb to the MME to transfer information for a transport network layer (TNL) association. The purpose of the S1 Setup procedure is to exchange application level data needed for the enodeb and MME to interoperate on the S1 interface. This procedure is the first S1-AP procedure triggered after the TNL association has become operational. The procedure uses non-ue associated signaling. The number of S1 Setup Response messages that the MME(s) sent to the enodeb. This message is sent by the MME to the enodeb to transfer information for a transport network layer (TNL) association. The number of S1 Setup Failure messages that the MME(s) sent to the enodeb. This message is sent by the MME to the enodeb to indicate a failure of the S1 Setup procedure. The number of Initial UE Message messages received from the enodeb. The purpose of the NAS Transport procedure is to carry UE/MME signaling over the S1 Interface. When the enodeb has received from the radio interface the first UL NAS message transmitted on an RRC connection to be forwarded to an MME, the enodeb invokes the NAS Transport procedure and sends the Initial UE Message message to the MME including the NAS message as a NAS-PDU IE. The number of DOWNLINK NAS TRANSPORT messages that the MME(s) sent to the enodeb. If the MME sends a NAS message transparently via the enodeb to the UE and a UE-associated logical S1- connection exists for the UE or if the MME has received the enb UE S1AP ID IE in an INITIAL UE MESSAGE message, the MME sends a DOWNLINK NAS TRANSPORT message to the enodeb including the NAS message as a NAS-PDU IE. If the UE-associated logical S1-connection is not established, the MME allocates a unique MME UE S1AP ID to be used for the UE and includes that in the DOWNLINK NAS TRANSPORT message. The number of UPLINK NAS TRANSPORT messages received from the enodeb. When the enodeb receives a NAS message from the radio interface to be forwarded to the MME to which a UE-associated logical S1-connection for the UE exists, the enodeb sends the UPLINK NAS TRANSPORT message to the MME, including the NAS message as a NAS-PDU IE Cobham

29 Appendices TeraVM available performance statistics Rx NAS Non Delivery Tx Initial Context Setup Rx Initial Context Response Rx Initial Context Setup Response Fail Rx UE Context Release Request Tx UE Context Release Command Rx UE Context Release Complete Tx Erab Setup Request Rx Erab Setup Response Rx Erab Release Indication Tx Erab Release Command Rx Erab Release Response The number of NAS NON DELIVERY INDICATION messages received from the enodeb. This message is sent from the enodeb to the MME when the enodeb decides to not start the delivery of a NAS message that has been received over a UEassociated logical S1-connection; or the enodeb is unable to ensure that the message has been received by the UE. The number of Initial Context Setup Request messages that the MME(s) sent to the enodeb. This message is sent by the MME to the enodeb to request the setup of a UE context. The purpose of the Initial Context Setup procedure is to establish the necessary initial UE Context, including ERAB context; the Security Key; Handover Restriction List; UE Radio capability; UE Security Capabilities; and so forth. The procedure uses UE-associated signaling. The number of Initial Context Setup Request messages received from the enodeb. This message is sent by the enodeb to the MME to confirm the setup of a UE context. The number of Initial Context Setup Failure messages received from the enodeb. The number of UE Context Release Request messages received from the enodeb. This message is sent by the enodeb to the MME to request the release of the UEassociated S1-logical connection over the S1 interface. The number of UE Context Release Command messages that the MME(s) sent to the enodeb. This message is sent by the MME to the enodeb to request the release of the UE-associated S1-logical connection over the S1 interface. The number of UE Context Release Complete messages received from the enodeb. This message is sent by the enodeb to the MME to confirm the release of the UE-associated S1-logical connection over the S1 interface. The number of E-RAB Setup Request messages that the MME(s) sent to the enodeb. This message is sent by the MME to the enodeb to request the assignment of resources on Uu and S1 for one or several E-RABs. The number of E-RAB Release Response messages received from the enodeb. This message reports the outcome of the request made by the E-RAB Release Request message. The number of E-RAB Release Indication messages received from the enodeb. This message is sent by the enodeb to indicate that the MME should release one or several E-RABs for one UE. The number of E-RAB Release Command messages that the MME(s) sent to the enodeb. This message is sent by the MME to the enodeb to release allocated resources on Uu and S1 for one or several E-RABs. The number of E-RAB Setup Response messages received from the enodeb. This message reports the 2015 Cobham

30 Appendices TeraVM available performance statistics Rx S1AP Error Indications Tx S1AP Error Indications Emergency High Priority MT Access MO Signaling Total Attach Initiated Total Attach Succeeded Total Attach Failed EPS Attach Initiated EPS Attach Succeeded EPS Attach Failed Combined EPS/IMSI Attach Initiated Combined EPS/IMSI Attach Succeeded Combined EPS/IMSI Attach Failed EPS Emergency Attach Initiated EPS Emergency Attach Succeeded EPS Emergency Attach Failed Rx Attach Request outcome of the request made by the E-RAB Setup Request message. The number of Error Indication messages that the MMEs received from the enodeb. The Error Indication message is sent by both the MME and the enodeb to indicate that some error has been detected in the node. The number of Error Indication messages that the MME(s) has sent to the enodeb. The Error Indication message is sent by both the MME and the enodeb to indicate that some error has been detected in the node. The RRC Establishment Cause IE in the S1AP Initial UE Message is "Emergency". This indicates the establishment or existence of an Emergency session. The RRC Establishment Cause IE in the S1AP Initial UE Message is "High Priority". This indicates the UE Access Class is in the range of 11 through 15. The RRC Establishment Cause IE in the S1AP Initial UE Message is "MT Access". This can occur if the UE Exits the IDLE state due to a Paging message. The RRC Establishment Cause IE in the S1AP Initial UE Message is "MO Signaling". This message can be sent for the following NAS procedures when the UE is in the IDLE state: Attach, Detach, Service Request, Extended Service Request, TAU. The total number of Attach procedures (of all types) that were initiated during the test. The total number of Attach procedures (of all types) that were successfully completed during the test. The total number of Attach procedures (of all types) that were attempted but failed during the test. The total number of EPS Attach procedures that were initiated during the test. The total number of EPS Attach procedures that were successfully completed during the test. The total number of EPS Attach procedures that were attempted but failed during the test. The number of combined EPS/IMSI Attach procedures initiated. This statistic increments for CSFB activities. The number of combined EPS/IMSI Attach procedures that were successfully completed. This statistic increments for CSFB activities. The number of combined EPS/IMSI Attach procedures that were attempted but failed. This statistic increments for CSFB activities. The number of Emergency Attaches that were initiated. The number of Emergency Attaches that were successfully completed. The number of Emergency Attaches that were attempted but failed. The number of Attach Requests received from the UE, as part of an attach procedure Cobham

31 Appendices TeraVM available performance statistics Tx Attach Accept Rx Attach Complete Tx Attach Reject Rx PDN Connectivity Request Tx Activate Default Bearer Request Rx Activate Default Bearer Accept Tx PDN Connectivity Reject Emergency Sessions Initiated Emergency Sessions Succeeded Emergency Sessions Failed Emergency Sessions Active Emergency Max Sessions Active EPS Emergency Attach IMSI Initiated EPS Emergency Attach IMSI Succeeded EPS Emergency Attach IMSI Failed EPS Emergency Attach IMEI Initiated EPS Emergency Attach IMEI Succeeded EPS Emergency Attach IMEI Failed EPS Emergency Attach GUTI Initiated EPS Emergency Attach GUTI Succeeded EPS Emergency Attach GUTI Failed High Priority Sessions Initiated High Priority Sessions Succeeded The number of Attach Accept messages sent from the network to the UE. This message is sent to the UE to indicate that the corresponding attach request has been accepted. The number of Attach Complete messages received from the UE. This message is sent by the UE in response to an Attach Accept message. The number of Attach Reject messages that the network sent to the UE. This message is sent to the UE to indicate that the corresponding attach request has been rejected. The number of PDN Connectivity Request messages received from the UE. This message requests establishment of a PDN connection. The number of Activate Default EPS Bearer Context Request messages sent to the UE. This message requests activation of a default EPS bearer context. The number of Activate Default EPS Bearer Context Accept messages received from the UE. This message acknowledges activation of a default EPS bearer context. The number of PDN Connectivity Reject messages sent to the UE. This message indicates that the request for establishment of a PDN connection has been rejected. Sessions Initiated with PDN connection type of Emergency. Sessions Succeeded with PDN connection type of Emergency. Sessions Failed with PDN connection type of Emergency. Active Sessions with PDN connection type of Emergency. The maximum number of sessions with PDN connection type of Emergency. Emergency Attaches Initiated with UE identity type IMSI. Emergency Attaches Succeeded with UE identity type IMSI. Emergency Attaches Failed with UE identity type IMSI. Emergency Attaches Initiated with UE identity type IMEI. Emergency Attaches Succeeded with UE identity type IMEI. Emergency Attaches Failed with UE identity type IMEI. Emergency Attaches Initiated with UE identity type GUTI. Emergency Attaches Succeeded with UE identity type GUTI. Emergency Attaches Failed with UE identity type GUTI. Session initiated for which the S1AP Initial UE Message indicates an RRC Establishment cause of High Priority. Session succeeded for which the S1AP Initial UE Message indicates an RRC Establishment cause of High Priority Cobham

32 Appendices TeraVM available performance statistics High Priority Sessions Failed UE Initiated UE Succeeded UE Failed NW Initiated NW Succeeded NW Failed NW Canceled NW Skipped UE Deactivation Initiated UE Deactivation Succeeded UE Deactivation Failed NW Deactivation Initiated NW Deactivation Succeeded NW Deactivation Failed NW Deactivation Canceled NW Deactivation Skipped Active Max Bearers Terminated Session failed for which the S1AP Initial UE Message indicates an RRC Establishment cause of High Priority. The total number of UE-initiated dedicated bearer Attach Requests that were initiated. The total number of UE-initiated dedicated bearer Attach Requests that were successfully completed. The total number of UE-initiated dedicated bearer Attach Requests that were rejected. The total number of network-initiated dedicated bearer Attach Requests that were initiated. The total number of network-initiated dedicated bearer Attach Requests that were successfully completed. The total number of network-initiated dedicated bearer Attach Requests that were rejected. The number of network-initiated dedicated bearers for which the bearer setup was canceled. The total number of UE-initiated dedicated bearer Attach Requests that were skipped, for either of the following reasons: - There was an internal Database look-up error (in which case the UE cannot be found). - The UE is in the IDLE state when IxLoad is ready to send the message to the enodeb (in which case there is no valid S1 context). The number of UE-initiated dedicated bearers for which tunnel deactivation was initiated. The number of UE-initiated dedicated bearers for which tunnel deactivation succeeded. The number of UE-initiated dedicated bearers for which tunnel deactivation failed. The number of network-initiated dedicated bearers for which tunnel deactivation was initiated. The number of network-initiated dedicated bearers for which tunnel deactivation succeeded. The number of network-initiated dedicated bearers for which tunnel deactivation failed. The number of network-initiated dedicated bearers for which the bearer release was canceled. The number of UE-initiated dedicated bearers for which tunnel deactivation was skipped, for either for the following reasons: There was an internal Database look-up error (in which case the UE cannot be found). The UE is in the IDLE state when IxLoad is ready to send the message to the enodeb (in which case there is no valid S1 context). The total number of dedicated bearers that are currently active. This is a real-time statistic, rather than a cumulative total. The maximum number of dedicated bearers that have been active at any one time during execution of the test. The total number of dedicated bearers that were terminated Cobham

33 Appendices TeraVM available performance statistics Dedicated Bearer Failed, Internal Error Dedicated Bearer Failed, No Matching TFT Dedicated Bearer Failed, QCI Mismatch Dedicated Bearer Failed, Requested QOS Does Not Fit Dedicated Bearer Failed, UE Not Connected Dedicated Bearer Failed, LBI Not Found Dedicated Bearer Failed, Bearer Already Exists Dedicated Bearer Failed, All Bearers Activated GTP-u packets/s Tx GTP-u packets/s Rx GTP-u packets/s Reflected GTP-u packets/s Rerouted GTP-u kbps Tx GTP-u kbps Rx GTP-u kbps Reflected GTP-u kbps Rerouted GTP-u kbps Tx There was an internal error that resulted in the bearer being rejected. No bearers configured for this APN matched the requested TFT. A bearer with matching TFT was found on the requested APN and the requested QOS fits, but the QCI did not match. A bearer with matching TFT was found on the requested APN, but the requested QOS does not fit. The UE requested bearer failed because the UE was not in the connected state. The UE requested bearer failed because no matching session with the Linked EPS Bearer Identity (LBI) was found. The UE requested dedicated bearer failed because it is already active. All 11 bearers are already active for this UE. The rate (in packets per second) at which GTP-U packets are transmitted, for all bearers, across the S1-U interface. The rate (in packets per second) at which GTP-U packets are received, for all bearers, across the S1-U interface. The count of data plane traffic sent on indirect tunnels, measured in packets per second. When an S1-based handover is executed, a new tunnel is created for indirect forwarding, where the downlink packets are reflected back to the SGW during the handover process. Once the handover is complete, the SGW transmits those reflected packets to the target enodeb. The rate (in packets per second) at which GTP-U packets have been forwarded between tunnels, on the server (MME) side. The rate (in kilobits per second) at which GTP-U traffic is transmitted, for all bearers, across the S1-U interface. The rate (in kilobits per second) at which GTP-U traffic is received, for all bearers, across the S1-U interface. The count of data plane traffic sent on indirect tunnels, measured in kilobits per second. When an S1-based handover is executed, a new tunnel is created for indirect forwarding, where the downlink packets are reflected back to the SGW during the handover process. Once the handover is complete, the SGW transmits those reflected packets to the target enodeb. The rate (in kilobits per second) at which GTP-U packets have been forwarded between tunnels, on the server (MME) side. The data rate (in kbps) at which GTP-U packets are transmitted, for all bearers, across the S1-U interface Cobham