Quality Assurance Manual for. Billing and Metering Integrity Scheme

Transcription

1 HKTA 3104 Issue 2 December 2005 Quality Assurance Manual for Billing and Metering Integrity Scheme TELECOMMUNICATIONS AUTHORITY HONG KONG

2 FOREWORD 1. On 30 May 2000, the Office of the Telecommunications Authority (OFTA) issued a Consultation Paper on Billing and Metering Accuracy of Public Telecommunications Services in Hong Kong proposing to implement a scheme that would best suit the need of customers and the industry and at the same time be economically and technically feasible for implementation in Hong Kong s environment. 2. Having considered the submissions from the industry on the above Consultation Paper, the Telecommunications Authority (TA) issued a Statement on 4 August 2000 to set out his considered views and intention that there is a genuine need to introduce and implement such a scheme in Hong Kong for enhancement of the confidence of the customers on billing and metering accuracy of the industry. In order to ensure early implementation of the proposed scheme, OFTA set up an industry forum on 15 September 2000 with participation of representatives from the concerned telecommunications operators and their associations, the Consumer Council and user groups to devise the details of the proposed scheme called Billing and Metering Integrity Scheme (BMIS) which includes the billing and metering integrity standards, self-appraisal system, assurance system, reporting system, monitoring system and implementation plan. 3. The Quality Assurance Manual has been developed to specify the required metering and billing integrity standards as well as the self-appraisal requirements and procedures to be followed and complied by the industry. Apart from this document, another major document called HKTA 3105, Requirements on the Assurance, Reporting and Monitoring Procedures for the Billing and Metering Integrity Scheme listing the general requirements and procedures on the assurance, reporting and monitoring has also been developed in order that the industry could follow and observe these requirements and procedures closely in compliance with BMIS. 4. All the HKTA specifications, consultation papers, statements and information notes issued by the Telecommunications Authority related to the BMIS can be obtained by downloading them from the OFTA's Internet website: HKTA 3104 Issue 2 Page 2

3 5. Any enquiry or request for further information regarding this manual can be addressed to :- Senior Telecommunications Engineer (Regulatory Section 11) Office of the Telecommunications Authority 29/F, Wu Chung House, 213 Queen's Road East, Wanchai, Hong Kong. Fax: HKTA 3104 Issue 2 Page 3

4 AMENDMENT TABLE Item Issue No. Paragraph Descriptions 1 2 Foreword Update the contact point The interval of submission of self-declaration forms is revised from quarterly to half-yearly Ditto Ditto Ditto 6 2 Section 10 Update the reference documents HKTA 3104 Issue 2 Page 4

5 Table of Contents 1. Introduction 6 Page 2. Glossary 7 3. Scope of the Manual 9 4. Billing and Metering Integrity Standards 9 5. Telecommunications Acceptance Models Telecommunications Metering Acceptance Model Telecommunications Billing Acceptance Model Testing Tolerance Levels Metering Tolerance Billing Tolerance Rounding Treatments Verification of Metering and Billing System Metering Test Billing Test Pre-paid Services Guidelines to Design Operator s Detailed Operation Manual Introduction Details of Each Section Monthly Test Report Introduction Self-Declaration Procedures Format of Test Report Reference Documents 49 HKTA 3104 Issue 2 Page 5

6 Annex 1 Switching Rules and Procedures 50 Annex 2 Self-Declaration Form for Compliance Status 51 Annex 3 Self-Declaration Form for the Metering Test 52 Annex 4 Self-Declaration Form for the Billing Test 53 Annex 5 Format of Metering and Billing Test Report 54 Annex 6 Inspection Status 62 Annex 7 Format of Test Report for Rounding Treatment Test 63 HKTA 3104 Issue 2 Page 6

7 1. Introduction 1.1 The main purpose of this Quality Assurance Manual (QAM) is to provide a quality assurance framework such that the operators can implement their self-appraisal systems in a timely and responsive manner in compliance with the BMIS. It is hoped that this manual would also facilitate the understanding of the general public on what controls and measures are to be implemented by operators to assure the accuracy and integrity of their bills issued to customers. 1.2 This manual is published, distributed and maintained by OFTA. Apart from this manual, the operators are required to develop their own Detailed Operation Manual (DOM) and perform measurements or generate Test Calls in accordance with the requirements and procedures stipulated in this manual. 1.3 If necessary, OFTA or OFTA s appointed agent will perform an on-site audit on operators Billing and Metering Systems. 1.4 Operators are encouraged to implement ISO 9001: 2000 Quality Management Systems - Requirements in their Billing and Metering Systems so that a well-established process control mechanism can be developed. However, the compliance or adoption of this ISO requirement by operators is voluntary and up to operators individual commercial decision and operational requirements. If necessary, OFTA may conduct a review and consultation with operators on whether there is a genuine need for operators to strengthen the process control of billing and metering systems by generally adopting this and/or other related ISO requirements. HKTA 3104 Issue 2 Page 7

8 2. Glossary Acceptance Number The number that is used for determining monthly test acceptability, i.e. the number of failed samples allowable in a test. Acceptance Quality Limit (AQL) Billing Billing Tolerance Call Detail Records (CDR) Call Duration Called Party Calling Party Certified Public Accountant (CPA) Charged Party Charging Time Stamp Delay Quality level that is the worst tolerable process average when a continuing series of lots is submitted for acceptance sampling. In BMIS, AQL refers to the quality level to show the likely percentage of failed items in a monthly Metering/Billing Test. Billing means the function of public telecommunications operators or service providers in: - assembling the charges incurred by a customer during the billing period; - applying any debts or credits outstanding or discounts due and calculating the net amount to be paid by the customer; and - issuing the bill. The maximum deviation allowed in each call charge of Billing Test. Records of detailed information such as call type, call start time, call duration, calling party, called party etc captured by the switches. The interval between the call is actually established that begins when the call is answered and ends when the call is terminated. A called party is the person who receives the call. A calling party is the person who originates the call. Certified Public Accountant means a professional accountant registered as such by virtue of section 24 of the Professional Accountants Ordinance (Chapter 50) and holding a valid and current practising certificate. A charged party is the person registered with the telecommunications operator or service provider concerned responsible for the payment or settlement of a specific or group of bills. Charging time stamp is the time recorded in a CDR used for calculating the charge amount for the services provided or consumed and the applicable tariff thereto. The time required for a signal to transit from the sending point to the receiving metering point. HKTA 3104 Issue 2 Page 8

9 Metering Metering Tolerance Per Call Test Switching Rules Test Call The process of capturing and recording call details such as calling number, time of day, call duration etc. for billing purpose. The maximum deviation allowed for call duration in the Metering Test. A two-part Metering Test: Part 1 to reconcile Test Call log records with call details from a metering point and Part 2 to reconcile call details from that metering point with call details from Billing System. A set of rules to control the switching of status between Normal, Reduced, Tightened and Discontinue Inspection. A call generated manually or automatically to verify the functioning and accuracy of a Metering System. HKTA 3104 Issue 2 Page 9

10 3. Scope of the Manual This manual describes the billing and metering integrity standards, the sampling models, metering / billing tolerance levels, the verification procedures and the format of the test report as required by the BMIS. It also provides guidance for operators to design their own DOM. 4. Billing and Metering Integrity Standards The billing and metering integrity standards of public telecommunications services are described as below:- Metering Integrity Standard Number of inaccurately metered calls shall not exceed 1 in 10,000 of total calls. Billing Integrity Standard Absolute summation error value of the inaccurately billed calls in a bill shall not exceed 1 in 10,000 of the value of total billed sum in that bill. HKTA 3104 Issue 2 Page 10

11 5. Telecommunications Acceptance Models 5.1 Telecommunications Metering Acceptance Model (the Metering Model) Introduction The sampling model for Metering System is developed in accordance with ISO : 1999(E) Sampling Procedures for Inspection by Attributes Part 1: Sampling Schemes Indexed by Acceptance Quality Limit (AQL) for Lot-by-Lot Inspection on the condition that live traffic should be used in the first place. AQL is a quality level to show the likely percentage of failed items in a monthly Metering Test. This manual has made use of the same or similar terms given in the above ISO document. If there were practical and technical limitations such that live traffic could not be used, operators could use Test Calls that should be conducted in the same environment and conditions of live traffic within the concerned operators networks The purpose of the monthly Metering Test is to examine whether the metering integrity standard of 1:10,000 given in Section 4 is met and to ensure an acceptable quality level in metering accuracy for a specific traffic in a month. Operators shall start from the normal inspection. Normal, tightened or reduced inspection shall continue to be unchanged on successive monthly Metering Tests, except where the Switching Rules require changes. The Switching Rules shall be applied to each class of non-conformities or nonconforming units independently. Number of Test Calls for Metering Test The Telecommunications Metering Acceptance Model (the Metering Model) governs the acceptance criteria and the amount of Test Calls to be conducted by operators of Fixed Telecommunications Network Services (FTNS), mobile network services (including services provided by Public Mobile Radiotelephone Services Licensees, Personal Communications Services Licensees and Mobile Carrier Licensees), Public Non-exclusive Telecommunications Services (PNETS) (for example, External Telecommunications Services (ETS), International Calling Card Services, Virtual Private Network Services, International Simple Resale (ISR) for Facsimile and Data (ISR fax and data), International Value-Added Network Services and Internet Services Providers (ISP), Mobile Virtual Network Operators (MVNO) etc.). The Metering Model follows some of the requirements in ISO :1999(E) with the details given in the HKTA 3104 Issue 2 Page 11

12 following table: Number of Test Calls Per Month Per Licence (Note 1) Reduced Normal Tightened Acceptance No / AQL Inspection Inspection Inspection Rejection No FTNS/Mobile/PNETS , % Table 1 : Telecommunications Metering Acceptance Model Note 1 : Each mobile network operator is required to conduct a number of Test Calls stated in Table 1 irrespective of number of licences issued to that particular operator for operation of second generation mobile services (i.e. GSM 900/1800, TDMA and CDMA technology). Switching Rules The diagram showing details of the Switching Rules is attached in Annex 1. Normal to Tightened Under the normal inspection, operators are required to sample or generate a specific number of live or Test Calls every month in accordance with the requirements in Table 1. When two monthly Metering Tests out of five consecutive monthly Metering Tests (or fewer than five consecutive monthly Metering Tests) have been non-acceptable on original inspection, operators are required to implement the requirements of the tightened inspection. Tightened to Normal Under the tightened inspection, operators are required to sample or generate a specific number of live or Test Calls every month in accordance with the requirements in Table 1. When five consecutive monthly Metering Tests that have been considered acceptable on original inspection, operators are allowed to switch to the normal inspection. Normal to Reduced Under the normal inspection, operators are required to sample or generate a specific number of live or Test Calls every month in accordance with the requirements in Table 1. Operators can switch to the reduced inspection provided that all of the following HKTA 3104 Issue 2 Page 12

13 conditions are satisfied: a) the current value of the switching score (see Section 5.1.8) is at least 30; and b) reduced inspection is considered desirable by the operator. Remark: the ISO s criterion of production is at a steady rate is not applied to the Metering Model when switching from the normal inspection to the reduced inspection. Switching Score The calculation of the switching score shall be initiated at the start of normal inspection. The switching score shall be set at the zero value at the start and updated following the inspection of each subsequent monthly Metering Test on original normal inspection. When the acceptance number is 0, add 2 to the switching score if a specific monthly Metering Test is accepted; otherwise reset the switching score to zero. For example, the switching score is 2 when the monthly Metering Test in, say, May 2002 is passed. The switching score is 4 when monthly Metering Tests in May 2002 and June 2002 are passed. The switching score is reset to zero if the monthly Metering Test in July 2002 failed. Reduced to Normal Under the reduced inspection, operators can switch to the normal inspection if any of the following conditions occur on original inspection: a) a monthly Metering Test is not accepted; or b) instructions given by the TA that normal inspection shall be re-instated. Remark: ISO s criterion of production becomes irregular or delayed is not applied to the Metering Model when switching from the reduced inspection to the normal inspection. Discontinuation of Inspection If the cumulative number of monthly Metering Tests not accepted in a sequence of consecutive monthly Metering Tests on original tightened inspection reaches 5, the acceptance procedures from Section to Section shall not be resumed until the HKTA 3104 Issue 2 Page 13

14 operator concerned has already improved its quality assurance system in a manner to the satisfaction of the TA. Tightened inspection shall then be used as if Section had been invoked. HKTA 3104 Issue 2 Page 14

15 5.2 Telecommunications Billing Acceptance Model (the Billing Model) The Billing Model is also based on ISO :1999(E). Operators are required to perform a Billing Test each month by sampling a number of bills specified with respect to the applicable status in Table 2 below. The purpose of the monthly Billing Test is to examine whether the billing integrity standard of 1:10,000 given in Section 4 is met and to ensure an acceptable quality level of billing accuracy for the bills issued in a month. Number of Bills to be Sampled Each Month Acceptance No / AQL Rejection No Sample 32 bills in the reduced inspection 0, % Sample 50 bills in the normal inspection Sample 80 bills in the tightened inspection Table 2 : Telecommunications Billing Acceptance Model A specific monthly Billing Test is considered to be not acceptable if the individual total error sum over the total billed sum of any sampled telephone bill is greater than 1/10,000. Normal to Tightened The Switching Rules as specified in Annex 1 are also applicable to the Billing Model. Under the normal inspection, operators are required to sample a specific number of bills every month in accordance with the requirements in Table 2. When two monthly Billing Tests out of five consecutive monthly Billing Tests (or fewer than five consecutive monthly Billing Tests) have been non-acceptable on original inspection, operators are required to implement the requirements of the tightened inspection. Tightened to Normal Under the tightened inspection, operators are required to sample a specific number of bills every month in accordance with the requirements in Table 2. When five consecutive monthly Billing Tests which have been considered acceptable on original inspection, operators are allowed to switch to the normal inspection. Normal to Reduced HKTA 3104 Issue 2 Page 15

16 5.2.5 Under the normal inspection, operators are required to sample a specific number of bills every month in accordance with the requirements in Table 2. Operators can switch to the reduced inspection provided that all of the following conditions are satisfied: a) the current value of the switching score (see Section 5.2.6) is at least 30; and b) reduced inspection is considered desirable by the operator. Remark: the ISO s criterion of production is at a steady rate is not applied to the Billing Model when switching from the normal inspection to the reduced inspection. Switching Score The calculation of the switching score shall be initiated at the start of the normal inspection. The switching score shall be set at the zero value at the start and updated following the inspection of each subsequent monthly Billing Test on original normal inspection. When the acceptance number is 0, add 2 to the switching score if a specific monthly Billing Test is accepted; otherwise reset the switching score to zero. For example, the switching score is 2 when the monthly Billing Test in, say, May 2002 is passed. The switching score is 4 when monthly Billing Tests in May 2002 and June 2002 also passed. The switching score is reset to zero if the monthly Billing Test in July 2002 failed. Reduced to Normal Under the reduced inspection, operators can switch to the normal inspection if any of the following conditions occur on original inspection: a) a monthly Billing Test is not accepted; or b) instructions given by the TA that normal inspection shall be re-instated. Remark: the ISO s criterion of production becomes irregular or delayed is not applied to the Billing Model when switching from the reduced inspection to the normal inspection. Discontinuation of Inspection If the cumulative number of monthly Billing Tests not accepted in a sequence of HKTA 3104 Issue 2 Page 16

17 consecutive monthly Billing Tests on original tightened inspection reaches 5, the acceptance procedures from Section to Section shall not be resumed until the operator concerned has improved its quality assurance system in a manner to the satisfaction of the TA. Tightened inspection shall then be used as if Section had been invoked. HKTA 3104 Issue 2 Page 17

18 6. Testing Tolerance Levels 6.1 Metering Tolerance Since the metering device of a network is usually not located at the customer end, the measured duration will have a small difference from that perceived by the customer. Therefore, it is necessary to introduce a term known as Metering Tolerance. Delay is the main cause of the Metering Tolerance The following are the Metering Tolerance for different services: Type of Services Tolerance IDD on Fixed Network Service Mobile Service (Note 2) Mobile Prepaid Service ETS Service ETS Calling Card Service Fixed Network Calling Card Service Modem Dial-up Internet Service/IVANS +/ ms +/ ms +/ ms +/ ms ms ms seconds Table 6.1 Metering Tolerance for Different Services Note 2: Mobile Service is applicable to mobile local calls, mobile IDD calls and mobile ETS calls Apart from Metering Tolerance for call duration, four more parameters need to be tested in the Metering Test. These parameters are Calling Party, Called Party, Charged Party and Charging Time Stamp No tolerance or error is allowed for recording of the Calling Party, the Called Party and the Charged Party. These parameters must be checked if they are relevant to the charging accuracy. Tolerance for the charging time stamp is +8.5 seconds and -5.5 seconds and this parameter needs to be checked if it is relevant to the charging accuracy. 6.2 Billing Tolerance During the Billing Test, call records from Billing System would be rerated by operators in accordance with the billing logic/tariff plan. This rerated per call charge will be compared with the per call charge appeared on the bill issued to the customer. No HKTA 3104 Issue 2 Page 18

19 tolerance will be given to the difference between the rerated per call charge and also the per call charge appeared on the bill issued to the customer. In other words, the billing tolerance of each billed call for different type of services is: IDD Service Mobile Service Modem Dial-up Internet Service : +/- 0 cent : +/- 0 cent : +/- 0 cent 6.3 Rounding Treatments All operators concerned are requested to declare their rounding treatments used in DOM and perform a rounding treatment test on all sampled bills in the Billing Test in the first twelve months after the implementation of the BMIS and report same in the manner specified herein. OFTA will conduct a review on whether there is a need to impose suitable accuracy requirements, integrity standards or restrictions on the rounding treatments or steps of individual operators The per call charge appearing in a bill will be subtracted from the actual per call charge calculated by using 4 decimal places in dollar value at each rating or mediation step. This difference is known as per call discrepancy (E ij ). All E ij will be summed up and reported to OFTA. E ij = A Bij - A Bij where i refers to i-th bill and j refers to j-th call of the i-th bill E ij refers to per call discrepancy A Bij refers to operator billed per call charge, calculated from operator s billing logic (usually with 2 decimal places in dollar value) A Bij refers to actual per call charge (with 4 decimal places in dollar value) When E ij is positive, it implies that it is to the benefits of the operator. When E ij is negative, it implies that it is to the benefits of the consumer. Operators shall specify the signs of E ij clearly in the results. For example, a telephone per call discrepancy is E ij = A Bij - A Bij = $0.01- $ = + $ Operators shall submit the results of the rounding treatment test to OFTA in both hardcopy and softcopy not later than 14 months and two weeks after the implementation HKTA 3104 Issue 2 Page 19

20 of the BMIS. The submitted test results would be studied by OFTA. The results of this rounding treatment test should be duly signed by the operator s responsible person, dated and chopped with the company chop. Operator s submissions of the test results should use the format/template given in Annex All submissions of hard copies of the rounding treatment test reports shall be addressed to: Senior Telecommunications Engineer (Technical Regulation 1) Office of the Telecommunications Authority 29/F, Wu Chung House, 213 Queen's Road East, Wanchai, Hong Kong At the same time, submissions of soft copies of the rounding treatment test reports shall be addressed to: meter@ofta.gov.hk HKTA 3104 Issue 2 Page 20

21 7. Verification of Metering and Billing System 7.1 Metering Test Operators are allowed to use automatic call generators to generate Test Calls from switches to test the metering accuracy. This section describes the two methods to perform the Metering Test. The operator may reconcile test log records from call generators with the corresponding records in the Billing System. The alternative method is to reconcile test log records from call generators with the records extracted from a metering point, followed by another reconcilation with the metering point and the corresponding records in the Billing System. In such case, the operator has to specify the metering point, say at the mediation device, in the DOM Nevertheless, manual Test Calls such as calling card services are acceptable if there is no appropriate test equipment for the operator. In this case, the operator may also reconcile manual test log records with the corresponding records in the Billing System. Methodology to Verify Metering System Using Test Calls The Metering Test The Metering Test is performed once every month and outlined in the diagram below: Test Metering Billing Bills Calls point System Test Call Log Comparison Call Details The test should involve the following steps: i) Generation of Test Calls ii) Collection of Test Call log iii) Collection of call details from the Billing System iv) Producing test result by comparing the Test Call log from call generator with the call details from the Billing System. Test Calls are accepted or rejected in accordance to the tolerance levels described in Section 6.1 HKTA 3104 Issue 2 Page 21

22 Number of Test Calls and Acceptance Level The number of Test Calls required in each test and the acceptance level are specified in Table 1. The distribution of numbers of Test Calls stated in Table 1 will be determined by applying either traffic ratio, revenue or number of customers of the operators concerned. In general, the operator must use the same consistent basis of either traffic ratio, revenue or number of customers in all pro-rata calculations for the numbers of Test Calls and Test Bills among relevant prepaid and postpaid services, and the use of pro-rata basis must be declared in DOM. Test Call Pattern In each Metering Test, operators should change test patterns quarterly in order to satisfy the randomness requirements on calling Directory Numbers (DNs), called DNs, charging DNs and call types. Directory Numbers Selection The operators should reserve a certain directory numbers (DN)s for the Metering Test. These DNs can be discrete spare numbers or belong to a specific number block. Operators can even use the internal network routing and testing numbers (i.e. numbers with the leading digits of 19 ) at their own convenience The operators have to use a pseudo random /ballot method to draw a calling DN and a called DN for the Metering Test. However, the calling DN and the called DN must be changed periodically so as to satisfy the randomness requirement. In case of dial-up Internet, the operators have to use a pseudo-random / ballot method to draw 2 log-in ID numbers from the reserved number list. 8 access numbers should be randomly selected from the available access number pool. If the pool is smaller than 8, all access numbers should be tested. In case of calling card calls, operators can select 2 cards from a reserved pool of 100 cards. Test Call Types The Test Call types are shown in the configuration diagrams at the end of this section. During the IDD test for ETS, mobile and fixed network services, operators are allowed to set up a loop back route to simulate the international call path. Operators may determine their own numbering scheme for performing this loop back test according to HKTA 3104 Issue 2 Page 22

23 their operational needs Regarding the indirect calls involved in fixed and mobile networks, there may be technical difficulties in setting up a test configuration across networks for indirect calls. Therefore, operators are allowed not to test indirect calls provided that they have tested the direct calls and could explicitly state and confirm that the call paths of indirect calls are having no significant difference to those of direct calls. Call Duration The call duration to be used in the Metering Test is: Call Type Test Call Duration (Minutes) IDD Calls 1.5 Mobile Calls 1.5 Dial-up Internet Calls 10 Tolerance of Measurement Equipment Operators may either use measurement equipment produced by equipment manufacturers in the market or tailor-made ones for their own use. However, they need to enclose the relevant certificate(s) by the equipment manufacturers or recognized laboratories or self-declaration on the accuracy/tolerance of such measurement equipment in use in the DOM in order that these certificates and self-declaration information will be audited by OFTA on need basis. The tolerance for measurement equipment is +/- 1 second. Maximum tolerance for measurement equipment for call duration is +/- 1 second. In case of manual call, the tolerance of manual calls is the same as Section The total tolerance for accepting the Test Calls is the tolerance of measurement equipment plus the Metering Tolerance mentioned in Section 6.1. Requirements on Test Execution Operators should follow the following requirements during the test execution: i) Operators need to check the integrity of the time stamps of the calls if the call types of those calls selected are in time zone tariff plans. Operators should ensure to continually align and adjust the time stamp of the Test Call logs HKTA 3104 Issue 2 Page 23

24 recorded by the call generators/receivers with the clock reference of the Hong Kong Observatory (time server stdtime.gov.hk ); ii) Operators should produce test reports and self-declaration forms according to the requirements described in Section 9 of this Manual; iii) Metering tolerances for these three parameters, that is, start time, end time and call duration, strictly follow Section 6.1 regardless of whether the parameters are directly obtained or indirectly derived from other parameters on the call record in the Billing System; iv) Total number of successful calls to be counted shall be no less than 99.9% of total Test Calls as specified in Table 1; and v) Operators may test more than the number of calls as specified in Table 1, but they shall use the valid calls (i.e. successful calls and those unsuccessful calls as specified in (iv) in sequence for checking and reporting purpose). Alternative Method by Per Call Test Operators have an option to perform the Metering Test in two steps according to their operational needs. However, the Metering Test is considered to have failed if either Part 1 test or Part 2 test failed. The details of the Part 1 test and Part 2 test are described in Section and respectively. Both tests are performed once every month. Part 1 Test The following diagram illustrates Part 1 of the test. Test Calls Metering Billing Bills Point System Test Call Log Comparison Part 1 Call Detailed Records This part is the same as that described in the Methodology to Verify Metering HKTA 3104 Issue 2 Page 24

25 System Using Test Calls of this section, except that the call details for comparison with the Test Call log are collected from the metering point output instead of the Billing System. The requirements specified in Section The Metering Test to Section Requirements on Test Execution are applicable to this part of the test. Part 2 Test Test Outline The objective of the test is to compare the call information of the CDR at the metering point output with the call details from the Billing System. Live calls are taken as samples for comparison. The details of the Part 2 test are illustrated in the following diagram. Live Calls Metering point Billing System Bills CDR Compariso Part 2 Call Details The test should involve the following steps: (i) Collection of CDR samples of live calls from metering point output on random basis; (ii) Collection of call details from the Billing System; and (iii) Producing test result by comparing the CDRs from the metering point and call details from the Billing System. Test Calls are accepted if the parameters set out in Section 6.1 are exactly matched. Number of Test Calls and Acceptance Number The number of Test Calls required in each test and the acceptance level are specified in Table 1. Call Sampling The calls are sampled according to a random selection of a calling number during a randomly selected period in the month when the Part 1 test is taken place. HKTA 3104 Issue 2 Page 25

26 Metering Test is not required for incoming toll free call, collect call and global conference call, etc if the Metering Systems are operated by other carriers or if the Metering System are manual. If necessary, OFTA may conduct a review in future for the inclusion of incoming toll free call, collect call and global conference call in the metering requirements. HKTA 3104 Issue 2 Page 26

27 Test Configurations (1/4) Call Generator Local Switch Toll Switch loop back Call Receiver Fixed Network CDR Fixed Network IDD Service Billing System HKTA 3104 Issue 1 Page 27

28 Test Configurations (2/4) Mobile Network Call Generator Base Station Base Station Controller MSC loop back (for IDD test only) Call Receiver Base Station Base Station Controller Mobile Network HKTA 3104 Issue 1 Page 28

29 Test Configurations (3/4) Option 1 Call Generator Call Receiver ETS platform with direct interface with Call Generator and Call Receiver ETS Platform loop back Option 2 Call Generator Call Receiver PSTN ETS platform cannot directly interface with Call Generator and Call Receiver. Test calls are passed to ETS platform via PSTN ETS ETS Platform loop back HKTA 3104 Issue 1 Page 29

30 Test C onfigurations (3/4) O ption 3 C all G enerator M obile N etw ork ETS platform loopback C all R eceiver Test C onfigurations (4/4) C all G enerator e.g. com puter PSTN ISP Platform R em ote A ccess Server R adius Server ISP HKTA 3104 Issue 1 Page 30

31 7.2 Billing Test Based on the sampled bills as mentioned in Section 5.2.1, operators are required to perform a Billing Test on a monthly basis by sampling 32, 50, 80 bills in the reduced, normal and tightened inspection respectively. The purpose of the Billing Test is to verify the billing accuracy of all sampled bills Operators will calculate the billing charges of calls by applying their own billing logic and using the call detail records information stored in their Billing Systems (A Bij ) which is exactly the same point of the Billing System used for Metering Test under Section (iii) or Section (iii). Afterwards, operators will compare the charges calculated from their billing logic (A Bij ) with the charges actually billed to customers (A Bij ). The absolute differences between these two charges in a bill will be added and regarded as total error in a bill (Σ D ij ) The actual bill amount is the summation of all chargeable values that the customer needs to pay to the operator concerned. For convenience purpose, the charge that the customer needs to pay to the operator is regarded as positive value, otherwise it will be regarded as negative value. Examples of negative value are free minutes, free call charges, balance brought forward in credit, odd cents carried forward etc Therefore, all the billing charges of positive value in a sampled bill will be added up and regarded as the actual bill amount (T i ) in a bill. All other negative values such as monetary rebates, free minutes, volume discount etc will not be counted in this amount. These billing charges include all timed/usage charges and non-timed/usage charges in that bill. Any billing charge of negative value will be checked arithmetically but will not be counted in the actual bill amount. When operators provide rebates in terms of time to their customers, the input information (time rebated from previous months or free minutes carried forward) can be assumed to be correct. Operators only need to perform the checking on the arithmetic calculation. When a sampled bill consists of negative chargeable values only, this sampled bill will be discarded and substituted by another sampled bill The passing criteria of an individual bill i is: ΣD ij / T i <= $1 / $10000 where i refers to a bill and j refers to a call T i refers the actual bill amount in a bill (down to cents place) D ij refers to per call error in a specific bill i and it equals to the absolute difference of ( A Bij - A Bij = ij A Bij and A Bij (D A Bij refers to operator billed per call charge calculated from operator s billing logic (down to cents place) A Bij refers to per call charge appeared on customer s bill (down to cents place) Arithmetic check based on charges actually billed to customers is counted as one of the passing criteria in a bill check HKTA 3104 Issue 1 Page 31

32 7.2.6 The monthly Billing Test is considered as failed if any individual sampled telephone bill is failed. The following table illustrates the number of bills to be sampled by operators each month and the acceptance/rejection criteria. Number of Bills to be Sampled Each Month Sample 32 bills in the reduced inspection Sample 50 bills in the normal inspection Sample 80 bills in the tightened inspection Acceptance No / Rejection No 0, However, operators shall ensure a random or pseudo-random selection of samples and also declare the selection process in the DOM. Operators should use one of the following parameters which would cover all their customers to randomize the selection of the samples. Random selection parameters are account number, directory number or invoice number Operators can use electronic records of the bills in the Billing System instead of hardcopy bills, provided that they are consistent with the bills actually issued to customers. Operators should have measures to ensure the consistency of the two records No Billing Test is required if the values of bills are calculated and checked by the other carriers. However, it is still necessary for operators to conduct an arithmetic check of these billing data to verify the correctness of the final bill value. If only raw data (e.g. call duration) is provided by other carriers, operators are still required to perform the associated Billing Test based on these raw data. Maximum Calls Per Bill Maximum calls per bill is a selection criteria in the Billing Test. If the number of calls of a sampled bill is greater than the maximum calls per bill, then this bill can be discarded. The maximum calls per bill for the Billing Test is defined as: IDD Mobile Dial-up Internet 35 calls per bill 191 calls per bill 34 calls per bill Minimum Call Per Bill The minimum call per bill in Billing Test is defined as 1. HKTA 3104 Issue 1 Page 32

33 7.3 Pre-paid Services Pre-paid services refer to the services provided by operators in the form of pre-paid calling cards and pre-paid SIM cards. Number of Test Calls Operators can use either traffic ratio, revenue or number of customers to determine the number of Test Calls (X) for the pre-paid services in the Metering Test whereas X is a portion of Test Calls in Table 1. For example, a FTNS operator is offering both post paid and pre-paid services and it decides to use traffic ratio to work out the number of Test Calls for post paid and pre-paid services. Assume its traffic ratio for post paid and pre-paid is 90% and 10% of total traffic respectively. Hence the number of Test Calls for post paid services are 1125 (1250 x 90%) and the number of Test Calls for the pre-paid services are 125 Test Calls (1250 x 10%). Number of Pre-paid Cards Similarly, operators can use either traffic ratio, revenue or number of customers to determine the number of pre-paid calling cards/pre-paid SIM cards (Y) for the Billing Test whereas Y is a portion of telephone bills in Table 2. For example, a FTNS operator is offering both post paid and pre-paid services and it decides to use traffic ratio to work out the number of pre-paid calling cards. Assume its traffic ratio for post paid and pre-paid services is 90% and 10% of total traffic respectively. Hence, the number of telephone bills for post-paid services are 45 (50 x 90%) and the number of pre-paid calling cards/telephone bills for the test are 5. In addition, half-adjustment rounding to the nearest integer must be applied in these pro-rata calculations, except that if the number for a prepaid service is smaller than 1, it must be rounded up to 1. Billing Integrity Standard As far as the Billing Test is concerned, the absolute summation error value of the inaccurately billed calls in a pre-paid calling card/pre-paid SIM card shall not exceed 1 in 10,000 of the value of total billed sum in that pre-paid calling card/pre-paid SIM card. Test Method HKTA 3104 Issue 1 Page 33

34 7.3.5 Automatic test for the pre-paid services is highly recommended and shall be adopted at a first place. However, if automatic generation of pre-paid calls is not feasible, then manual generation of Test Calls for pre-paid services is also acceptable. Tolerance of Call Start Time and Call Stop Time In case the call generation and call recording have to be undertaken manually, the tolerance of call start time or call end time is +/- 2 seconds. Accuracy of IVRS Voice Prompt Operators shall self-declare in DOM that the accuracy of the pre-paid calling card or pre-paid SIM card system in the conversion of electronic record to voice message. Scenario of Pre-paid Services Platforms In general, there are 2 scenarios for the pre-paid services systems. Scenario 1 refers to the pre-paid services system possessing a database in which the required call detailed information and the charge of each call have been recorded. Scenario 2 refers to the pre-paid services systems that do not have a database storing call detailed information and the charge of each call. Operators can select either separate or integrate test for the pre-paid service test. Separate Test The Metering Test is similar to Section 7.1. The following parameters should be checked, wherever applicable, such as calling party, called party, call duration, charged party (e.g. account number) and charging time stamp etc. The method of selecting Directory Numbers is set out in Section The number of pre-paid cards required for Metering Test is at least The Billing Test is similar to Section 7.2. All in-service pre-paid SIM/calling cards should be sampled for the Billing Test. In particular, procedures should be derived in such a way to ensure samples are chosen randomly or pseudo-randomly Sampling of calls/cards/bills can be made in either automatic way or manual ballot way. All the sampled records (calls, bills or cards etc.) must be listed out clearly irrespective of whether the calls/bills/cards were used, discarded or any other possible HKTA 3104 Issue 1 Page 34

35 uses However, if sampling of cards is a manual process, then all the cards drawn including filtered cards and selected cards must be recorded Only the calls made in previous month of the reporting period shall be included for the test. The maximum and minimum numbers of calls per card for the pre-paid system are the same as those per bill for post paid system. Integrated Test The charged party shall be verified when the Metering System is tested. Call charge should be calculated by applying the billing logic of the billing/pre-paid system. This call charge is verified against the actual call charge deducted by the prepaid system. The actual call charge deducted by the prepaid system can be checked and recorded by listening to the IVRS voice or other methods whereas appropriate In case the test is failed, operators shall conduct investigations, report the sources of faults (Metering System, Billing System, or both) and specify the remedy actions taken to rectify the faults to OFTA. In case call duration record is not retrievable from the Prepaid Service Platform, the operator must submit sufficient supporting information for its investigation and argument to declare that the fault is due to Metering System alone, Billing System alone, or both of them The passing criteria for pre-paid services using integrated platforms are similar to Section In case of pre-paid SIM cards offered by mobile operators which can make both IDD calls and mobile calls, the number of pre-paid IDD calls and pre-paid mobile calls are derived from the total number of pre-paid calls of that mobile operator using either traffic ratio, revenue or number of customers. The number of pre-paid cards required for integrated test is at least 2. General Procedure The general procedures for testing the prepaid systems are to: a) Check the prepaid cards for sufficient monetary value for making the Test Calls; b) Synchronize the clock with Hong Kong Observatory, if necessary; c) Record the balance at the beginning of the tests; d) Check the charged party (or card identifier) and duration during the Metering Test HKTA 3104 Issue 1 Page 35

36 whereas calling party, called party and time stamp would be checked if necessary; e) Record the remaining balance at the end of the tests; f) Record the before-call and after-call balances to obtain the per-call charge amount in case the per-call charge amount is not available from the prepaid system; g) Record the deviation of the charging amount calculated manually and the amount recorded by the prepaid system for each call; h) Add the absolute differences between manually calculated charge and the actual deducted amount for each call to be regarded as total error for the prepaid service; i) Sum all calculated chargeable values to become the actual bill amount; j) Check whether the passing criteria for the prepaid card is fulfilled that the ratio of the total error and the actual bill amount does not exceed 1: and that the arithmetic check set out in Section is passed. and k) Observe that the previous agreed procedures for handling rounding treatment for post paid service were applicable to prepaid service. HKTA 3104 Issue 1 Page 36

37 Template for Pre-paid Services Separate Test Metering Test Call type: [Optional Table] Account No. / Card Identifier Initial Card Value Derived Records from Test Calls Call Number Test Date Account No. / Card Identifier Called Number Answered Start Time Duration Derived from Prepay Service Platform Call Test Date Number Account No. / Card Identifier Called Number Answered Start Time Duration Test Result HKTA 3104 Issue 1 Page 37

38 Billing Test Account No. / Card Identifier Initial Card Value Call No X Test Date Called DN Start / End** Time (if applicable) Recorded Duration Total Rerated Call Charges Call Charges Recorded by the Billing System Card Balance* Absolute Error Note: * Card Balance, except the balance at the end of the test, is an optional item. ** Deleted if appropriate. HKTA 3104 Issue 1 Page 38

39 Integrated Test Call type : Account No. / Card Identifier Initial Card Value Call No* Test Called Date DN Card Balance Before Making the Test Call Start / End** Time (answered) [if applicable] Recorded Duration of Test Call Charging Unit * No of Unit Charge Units / Tariff (round up) * Manually Rated call Charges (X) Minimum Acceptable Duration * Maximum Acceptable Duration * Possible Call Charges (lower) * Possible Call Charges (upper) * Card Balance After Making the Test Call Call Charges Deducted by the Billing System Absolute Test Error (Y) Results (note) (metering) (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) (13) (14) (15) (16) (17) 1 $ $0.20 $ $1.80 $2.40 $97.80 $2.20 $ $ $0.20 $ $1.80 $2.40 $96.10 $1.70 $ $ $0.20 $ $1.80 $2.40 $93.50 $2.60 $ X Passing criteria for Billing Test Result is : summation of Y summation of X 1: 10,000 Billing Test Result : summation of y summation of x 1: 10,000 (pass criteria) Note: * Optional items ** Deleted if appropriate. (note) : If (12) <= (15) <= (13), the absolute error is 0. (i.e. Call No. 1 case) If (15) < (12), the absolute error is (12) - (15). (i.e. Call No. 2 case) If (15) > (13), the absolute error is (15) - (13). (i.e. Call No. 3 case) HKTA 3104 Issue 1 Page 39

40 8. Guidelines to Design Operator s Detailed Operation Manual 8.1 Introduction This section illustrates the guidelines to operators on how to design and compile their DOM. The DOM integrates the operator s all aspects of the Metering and Billing Tests from network operating system to back-office Billing Systems. It provides the information on the operator s internal testing and coordination procedures and system or equipment setup in meeting the requirements of this manual or the BMIS as a whole There are several guidelines to help operators to prepare the DOM such as: The purpose of the DOM is to give and explain the operation of operator s billing and metering processes, man machine interfaces, testing procedures and system or equipment setup. It could also serve to assist the CPA in understanding the detailed information before conducting the required assurance exercise; Apart from giving the operation of billing and metering processes, testing procedures and system/equipment setup, the DOM should also illustrate the details about the reporting channel and mechanism to OFTA; and DOM should also specify when and how the details given therein would be revised and updated The DOM should therefore include the following major headings: The description of the system/equipment setup to meet the requirements of the Metering Model and standards; The description of the system equipment setup to meet the requirements of the Billing Model and standards; The Metering Tests and yardsticks; The Billing Tests and yardsticks; Self appraisal procedures; Self declaration; Monthly Report; and Procedures for revision. 8.2 Details of Each Section Contents of DOM should include the details given in the following table: HKTA 3104 Issue 1 Page 40