REPORT ON THE 'MINI' REVIEW OF THE MONTHLY SURVEY TO INTERNET SERVICE PROVIDERS

Transcription

1 REPORT ON THE 'MINI' REVIEW OF THE MONTHLY SURVEY TO INTERNET SERVICE PROVIDERS SEPTEMBER 2005 A publication of the Government Statistical Service 1

2 CONTENTS Page EXECUTIVE SUMMARY...3 KEY FACTS INTRODUCTION BACKGROUND USES INTERNATIONAL COMPARISIONS RESPONDENTS' VIEWS COSTS RECOMMENDATIONS FOR CHANGE...11 Annex A...13 Annex B...17 Annex C...18 Annex D...23 Annex E...25 Any enquiries regarding this report should be addressed to: Mark Pollard Office for National Statistics Room Government Buildings Cardiff Road Newport NP10 8XG Tel: or Fax: For information on survey control, reviews and compliance 2 go to:

3 EXECUTIVE SUMMARY 1. The monthly survey of Internet Service Providers measures the number of subscriptions for Internet connections in the UK to produce the Index of Internet Connectivity. It was developed in response to a Department of Trade and Industry (DTI) White Paper to help monitor the UK's use of e- commerce and its progress towards making the UK 'the best environment in the world to do e-commerce'. 2. The survey is conducted on a statutory basis to a little more than 40 businesses. All UK Internet service providers (ISPs) come within scope of the survey and the panel of selected businesses is reviewed on a regular basis. 3. The Index of Internet Connectivity is used both within and outside government and is frequently used in conjunction with the Office of Communications (OFCOM) data, for example, to monitor the level of Internet connectivity in the UK. 4. All respondents to the survey were contacted as part of this review. In summary: Nine per cent of respondents that completed the review questionnaire had difficulty understanding the questions on the monthly questionnaire; Nine per cent of respondents had difficulty providing all data by the relevant date; All respondents found the guidance notes helpful; Eighty two percent would be able to provide a more detailed breakdown of their permanent subscriptions; Respondents and users had difficulty locating the published results on the ONS website. Recommendations 5. This review s recommendations include: Less detailed breakdowns are collected for dial-up subscriptions; More detailed breakdowns are collected for permanent subscriptions; The survey be conducted on a quarterly basis; Closer links should be forged with the Office for Communications (Ofcom); More detailed information should be included in the First Release; The future of the index should be reviewed again in 2006 to assess ongoing user needs. 3

4 KEY FACTS Date survey commenced January 2001 Statutory or voluntary Frequency Main information collected Statutory Monthly The survey collects data in respect of the number of active subscribers to the Internet for: Permanent connections; Dial-up connections; Type of subscription used for dial-up connections. Sample Size At the last selection prior to this review, the sample size was 44. Coverage Method As no standard international classification (SIC) exists for ISPs, selection of businesses is carried out using research from publications and on-line resources. Postal and survey Use of results: Primary Secondary Publication The primary use of this data is to measure the level of Internet connections compared to January 2001 in line with the original 2001 DTI goal mentioned above. There is no primary user for this data but it is widely reported in the press and used by the secondary users below. The data collected are used by Office of Communications (Ofcom), DTI, ISPs themselves and other commercial and academic users. The data are published as an index in a monthly first release, usually during the 7th week following the reference period. Compliance Cost The annual compliance for the survey was assessed as 12,873. ONS Cost The cost to ONS (at planning rates) to run the ISP survey during 2004/05 was estimated to be 28,104. This includes staffing and staff related costs, the provisions of computing facilities, general policy and methods support. 4

5 1. INTRODUCTION Reasons for Review 1.1 Standing instructions on the control of statistical surveys require that regular monthly surveys to businesses and local authorities be reviewed at least once every three years. The aim of these procedures is to avoid unnecessary surveys, to ensure that control is exercised in the most costeffective way and that authorised surveys are conducted in such a way as to obtain the information needed while imposing the minimum burden on respondents. 1.2 The present instructions for controlling Government statistical surveys, including the procedures for reviewing surveys, were issued by the Prime Minister's office in These instructions re-iterate the continuing need to review all regular surveys of business. Surveys with total compliance costs of 250,000 or more are subject to a comprehensive review involving an external observer. Smaller surveys are subjected to a less burdensome 'mini' review. The level of detail contained in the mini reviews is expected to reflect the complexity and importance of the survey and the level of compliance costs. As per office policy, a copy of this review has been sent to and commented on by the ONS' Quality Centre. 1.3 This review also covers a number of broader issues in addition to those in the standing instructions. Coverage of the review 1.4 This review covers: Monthly Survey of Internet Service Providers. The survey measures the number of subscriptions for Internet connections in the UK to produce the Index of Internet Connectivity. It was developed in response to a Department of Trade and Industry (DTI) white paper to help monitor the UK's use of e-commerce and its progress towards making the UK 'the best environment in the world to do e-commerce'. 5

6 2. BACKGROUND History of the survey 2.1 The survey began in 2001 and is directed at companies that provide connections to the Internet i.e. Internet Service Providers. 2.2 There have been few changes to the questionnaire since the survey began. In 2002, following consultation with ISPs, collecting the number of active subscriptions at the start and end of the collection period was dropped and replaced by the number of active subscriptions during the period. This prevented duplicate data being collected. A number of minor changes have also been made to clarify the definitions of the different access plans on the questionnaire. A copy of the current survey questionnaire is attached at Annex A. Data collection 2.3 From January 2005 data collection was carried out by the Business Data Division (BDD) of the ONS and results produced by the Financial and Accounting Services Division (FASD). Previous to January 2005, data collection and results production were carried out by FASD. 2.4 The survey collects the total number of active subscribers to the Internet during the reference period for: Total Permanent connections; Total Dial-up connections; Free/Billed dial-up connections; Unmetered dial-up connections; Mixed dial-up connections; Other connections. Surveys which ask for similar information 2.5 The ONS' e-commerce survey measures the percentage of businesses that have access to the Internet. It also breaks this measure down between narrowband and broadband connections as well as 'don't know'. These are collected annually on a statutory basis. 2.6 The ONS' Expenditure and Food survey measures the number of households that have access to the Internet at home. Also measured, as part of the National Statistics Omnibus Survey, is how individuals use the Internet and their attitude towards the Internet. Sample Design 2.7 The survey is based on a list of companies derived from various sources including trade websites. It is not possible to select businesses from the Interdepartmental Business Register (IDBR), as there is currently no Standard Industrial Classification (SIC) specific to ISPs. A review of SICs is taking place for 2007 and there are plans to include an SIC for ISPs. However, the ISP element of a business may be small, which would preclude many ISPs from the SIC and may also miss ISPs from selection if they change SIC. 2.8 The current sample is 44 business units. Operational procedures 2.9 The questionnaires are despatched at the end of the month and are addressed to a named contact within the company, requesting a response within 7 days of the despatch date. The survey issues 1 reminder letter following the return date. Telephone response chasing commences one week after the despatch of the reminder and continues until results are run The response target for this survey is 75% of questionnaires despatched, 6

7 in time for calculation of final results. A table of recently achieved response rates is attached at Annex B Businesses are selected as described in 2.7 above. Data is manually entered into an Ingres database where it is validated for potential errors These checks compare returned data with data returned for previous periods from the same contributor. Data that fail these checks are identified and queried with the contributor where necessary with the aim of editing or confirming the data. Any comments on the data are stored centrally allowing both collection and results team access, reducing the need to re-contact businesses. Results processing 2.13 Data are imputed for missing periods using normal linear interpolation techniques. Data are then aggregated for each category of connection type for each month of the survey. This monthly total is then divided by the number of subscribers in January 2001 and expressed as a percentage. Publication 2.14 Data from the survey are published as an index in the Internet Connectivity First Release on the ONS website. See 7

8 3. USES 3.1 The results of the monthly survey of Internet Service Providers are used both within and outside government. The following outlines the main uses of the data. ONS users 3.2 There are currently no ONS users of this data Other Government users 3.3 The index is used by DTI policy to monitor the growth of the Internet and the switch to broadband connections. 3.4 OFCOM use the index as a guide to the level of dial-up subscribers in the UK. They collect data on the number of asynchronous digital subscriber line (ADSL) and cable connections direct from the telecommunication operators. 3.5 Users were approached for their views as part of this review. They expressed interest in collecting additional broadband data and agreed that the survey should be collected quarterly. A full summary of the responses can be found at Annex E. Commercial Users 3.6 The monthly index is widely reported in the computer press and to a lesser extent in the general press. ISPs themselves use the data to compare their numbers and types of connections to the industry totals. 8

9 4. INTERNATIONAL COMPARISIONS 4.1 Data on the number of Internet connections is collected by many countries worldwide. The vast majority of these collect data from separate household and enterprise surveys and not directly from ISPs themselves. 4.2 Australia and Hong Kong are notable exceptions with data collected direct from ISPs. Data collected by these countries include number of connections and amount of data downloaded. Australia also produces their results at a regional level. 4.3 The data are collected on a mostly quarterly or annual basis although a few still collect on a monthly basis. 5. RESPONDENTS' VIEWS 5.1 A survey of contributors was conducted as part of this review, designed to estimate compliance cost and to seek views on the survey. Due to the small number of contributors to the monthly survey, all 44 were sent the triennial review questionnaire. A copy of the questionnaire can be found at Annex C. 5.2 After telephone response chasing, the ONS received 15 responses to the questionnaire, giving a response rate of 34% 5.3 Respondents reported that they found the guidance notes helpful. Some difficulties were reported in providing data for the survey. A detailed analysis of respondents' views can be found at Annex D. Complaints 5.4 There have been no contributor complaints about the Monthly Survey to Internet Survey Providers. 9

10 6. COSTS Government costs 6.1 The cost to Government to run the Monthly Survey to Internet Service Providers during 2004/05 was estimated to be 28,104 per annum. This includes staffing related costs, the provisions of computing facilities and general policy and methods support. Respondents' costs 6.2 The questionnaires despatched to respondents as part of this triennial review requested information on the time taken to complete the monthly questionnaire and the management level of the respondent. 6.3 The length of time taken to complete the survey questionnaire varied between 5minutes and 1 hour and 20 minutes. The median time taken to complete the survey was 15 minutes. Costs conclusion 6.4 Based on the 15 minute estimate the overall cost to business is now estimated to be 12,873 per annum. Reducing the burden on respondents 6.5 A variety of measures are used to reduce the burden on business. These include: If actual information is not readily available, then informed estimates are accepted. Re-contact with companies is minimised by the inclusion of a comments box on the questionnaire for respondents to record any unusual fluctuations in their data. A named contact in the ONS is given on the questionnaire and reminders. Information is stored centrally and is shared between the data collection and results teams. 10

11 7. RECOMMENDATIONS FOR CHANGE Periodicity Annex E Dial-up subscriptions Annex E Broadband/Permanent subscriptions Annex D/E 7.1 The survey should change from being collected and published on a monthly basis to a quarterly basis. This change would not affect users as the change from dial-up to broadband connections has been at a similar rate for some time. This will help save both ONS and contributor resources. We should aim to publish the quarterly data in a similar timescale to the current monthly index i.e. approximately 7 weeks after the end of the reference period. 7.2 The number of dial-up categories should be reduced to differentiate between unmetered and metered/other types of connection. This level of detail would still reflect the main types of dial-up connections and help reduce costs to respondents by reducing the number of questions. 7.3 The broadband/permanent category should be subdivided by speed of connection, as there are currently no alternative sources of data available. Possible categories being : Less than 512kbps; 512kbps to 1mbps; greater then 1mbps; Connection speed not known. The definitions should be clarified to include DSL, Cable, leased line and similar types of connection. This should enable better understanding of the questions by respondents. Additional data request Annex D/E Identifying ISPs for the survey Annex E Dissemination Annex D/E 7.4 Additional questions should not be added to cover the amount of data downloaded (bandwidth) and number of unsolicited (spam) s automatically blocked, due to the low number of businesses that are able to provide this data. Whether a business automatically scans e- mails for viruses, worms etc. should also not be added to the questionnaire, but it should be asked on an annual basis. 7.5 As there is currently no definitive population of ISPs, identifying new or significant ISPs is difficult and has led to revisions in the index when new ISPs have been added to the survey panel. Closer links should be forged with Ofcom with meetings being held on a quarterly basis. This would enable both departments to identify which businesses should be included in the survey. 7.6 Recommendations are: The index should continue to be published within the current timescale (approximately 7 weeks after the end of the reference period); We should publish subscription type data at 1 decimal place, when new broadband categories are introduced; Further research should take place to investigate publishing actual numbers of subscriptions when a satisfactory universe has been produced (see7.4); We should test the feasibility of publishing the relative proportions of household and business connections; Easier access should be given to back data with the possibility of including all back data on the ONS website; The website address of the Internet Connectivity press release should be included on the front of the monthly questionnaire. 7.7 The index should be rescaled to take into account the rapid increase in 11

12 the level of permanent/broadband connections. This should allow easier comparisons to be made between the 3 main series. The permanent/broadband index stood at over 5,000, in April 2005, when compared to its base of 100 in January 2001, and currently stands at over 10,000 whereas the dial-up index stands at less than 100. Future of the survey Annex E Implementing recommendations 7.8 The survey should continue, as there is no reliable source of the number of dial-up subscribers currently available. A further review of the survey should take place in 2006 to further examine the need for this survey. 7.9 The implementation of these recommendations is dependent upon the relevant resources being available and on any changes to compliance costs. 12

13 (ons) «contact» «Provider» «ta» «add1» «add2» «add3» «add4» «add5» «postcode» Annex A Monthly Survey to Internet Service Providers for the period February 2005 Notice is given under section 1 of the Statistics of Trade Act 1947 PLEASE USE BLACK INK TO COMPLETE THIS QUESTIONNAIRE Write any changes to your name/address in this box only 1 TO BE COMPLETED FOR: THE BUSINESS NAMED ABOVE FROM: Office for National Statistics A Government Executive Agency NEWPORT NP10 8XG YOUR CONTACT FOR HELP AND QUERIES: Fax xxxx David Norman Tel: xxxx xxxxx.xxxxxx@ons.gov.uk Please quote: «RU» PLEASE COMPLETE AND RETURN THIS QUESTIONNAIRE BY POST OR FAX BY: 7 th March 2005, for the calendar month of February 2005 Informed estimates are acceptable if exact figures are not available. Notes are enclosed to help you complete the form or you may telephone your contact named above. IMPORTANT: Failure to make a return can incur penalties under section 4 of the Statistics of Trade Act 1947 It is illegal for us to reveal your data or identify your business to unauthorised persons. FOR YOUR INFORMATION: The numbers of businesses and questions are kept to the minimum required to produce reliable results. This survey provides the government with information for monitoring the economy and compiling the national accounts. Copies of our code "Maintaining the Confidentiality of Data" are available on request. If you wish to use our minicom service for the Deaf please ring THANK YOU FOR YOUR CO-OPERATION 13

14 Background Notes 1. A virtual ISP is one which is serviced by a real ISP, for Internet access and interconnection. We need to make the distinction in order to avoid double-counting subscribers. 2. An active subscriber is a paying or non-paying customer to whom Internet access is provided by this ISP. The subscriber must have accessed the Internet in the month to which the survey refers. 3. Business subscribers include business, corporations, non-profit organisations and government organisations that obtain Internet access through this ISP. Non-business subscribers include private, household or residential customers. If you do not hold information on whether subscribers are business or non-business, please provide your best estimates. 14

15 1. Are you a virtual Internet Service Provider? (see note 1) please tick the box that applies Yes No 010 If yes, please give the name and address of your provider: Number of active subscribers at the end of the reporting period Business Non-business Dial-up subscribers Permanent connections Number of active subscribers at the end of reporting period, by type of access plan NOTE: total data in this section must equal total data in section 2 Business Non-business Free dial-up access (no direct payment from subscriber to ISP, call charges are paid to telecommunications supplier) Unmetered dial-up access (fixed price per month, quarter or year) Billing for dial-up call minutes (paid direct to ISP) Mixed dial-up access plan (fixed price plus call minutes) Fixed connection / always-on / broadband Other (Please specify)

16 Your comments and feedback on this questionnaire and survey Name of person to contact Position in company Tel No/Ext. Fax No Signature Date 16

17 Annex B Response Rates (per cent) Month At first publication date Current response January 2004 February 2004 March 2004 April 2004 May 2004 June 2004 July 2004 August 2004 September 2004 October 2004 November 2004 December 2004 January 2005 February Target: 75 per cent 17

18 Annex C Mark Pollard RITE Branch Office for National Statistics Cardiff Road Newport NP10 8XG 18 February 2005 Our Ref:«RU» Dear «contact» Triennial Review Of The Internet Service Provider Survey I am writing to ask you for your help with a review of the Internet Service Provider survey. The Office for National Statistics (ONS) regularly reviews each of its statistical inquiries as part of a Government initiative. You have been selected at random from a number of contributors whose businesses have participated in the Internet Service Provider Inquiry to take part in this review. Your response will make an important contribution to the review process. The information you provide will help us to assess the burden placed on contributors to the inquiry and ensure that these are kept to a minimum wherever possible. All the information you provide will be treated as confidential and neither you nor your business will be identified in reports on the review. Participation is voluntary but we hope you will help us by sharing your views on the Internet Service Provider survey. The enclosed questionnaire should be completed by the person who usually fills in the Internet Service Provider survey. The deadline for completion is 11 March 2005, if you prefer not to respond please let us know by returning the blank questionnaire in the enclosed stamped addressed envelope. The findings of the review will be available on the National Statistics website. If you have any questions about this survey or require any further information then please contact Jonathan Knight on tel: or fax: Alternatively, you can write to us at the address on the letterhead. Thank you in advance for your help. Yours Faithfully Mark Pollard Statistician 18

19 1. On average, how long does it take to complete the Internet Service Provider survey questionnaire? (please include any time spent extracting information from your accounting systems and collating data over and above normal operations). Hours Minutes 2. In your business, the person who usually completes the Internet Service Provider survey questionnaire is : Please 'X' one box only A Director A Senior Manager A Middle Manager A Junior Manager A Clerical Person Other (please specify) 3. Do you have difficulty understanding any of the questions on the Internet Service Provider survey questionnaire? Please 'X' one box only. Yes No Go to question 4 3a. For the question(s) you have difficulty understanding please write below, the question number(s) and a brief description of the difficulties that you have been experiencing: Question Number (s) Brief description of the difficulties that you have been experiencing 19

20 4. Overall, are the guidance notes helpful? Please 'X' one box only. Yes Go to question 5 No 4a. Please write below, any suggestions you have to make the guidance notes helpful: 5. Do you have any difficulties providing the data asked for on the Internet Service Provider survey questionnaire? Please 'X' one box only. Yes No Go to question 6 5a. Please write below, a brief description of the difficulties that you are having providing data for the Internet Service Provider survey: 6. Are you able to provide all the data by the specified return date? Please 'X' one box only. Yes Go to question 7 No 6a. Please write below, a brief description of why you cannot provide all the data by the specified return date: 20

21 7. If asked, would you be able to provide any of the following information on a monthly basis? For each category, you must "X" either yes or no Amount of data downloaded by subscribers in Gigabits (bandwidth) Yes No Number of subscribers to broadband and/or cable services by speed of connection (eg less than 512kbps; 512kbps to 1mbps; greater than 1mbps) Number of unsolicited (spam) s automatically blocked by your business Do you automatically scan s for viruses, worms and similar malware 7a. If you answer NO to any of the items in question 7 please provide a brief explanation why this data would not be available: 21

22 8. If you have any general comments about the Internet Service Provider survey questionnaire please write them below: Thank you for taking the time to complete this questionnaire. Please return the questionnaire in the pre-paid envelope by 11th March

23 Annex D Details and Analysis of Respondent Feedback On average how long does it take to complete the Internet Survey Provider survey questionnaire? The median time taken to complete the monthly questionnaire was 15 minutes with a mean time taken of 23 minutes. The amount of time taken each month varied between 5 minutes and 1 hour and 20 minutes. In your business, the person who usually completes the questionnaire is:? Thirty six per cent of monthly questionnaires were completed by a clerical person, with 27 per cent completed by a director, 18 percent by a senior manager with the remainder completed by either a middle manager or systems analyst. Do you have difficulty understanding any of the questions on the Internet Service Provider survey questionnaire? Nine per cent of respondents had difficulty understanding the questions on the monthly questionnaire. Overall, are the guidance notes helpful? All respondents found the guidance notes helpful. Do you have any difficulty providing the data asked for on the questionnaire? Nine per cent of respondents had experienced difficulties in providing data for the survey. Comments: "Our systems do not allow for easy extraction of the data required." Are you able to provide all the data by the specified return date? Nine per cent of respondents had problems supplying the required data by the date specific on the monthly questionnaire. Comments: "Dial-up usage isn't available to us immediately." If asked would you be able to provide any of the following information on a monthly basis: Amount of data downloaded by subscribers in Gigabits (bandwidth)? Twenty seven per cent of respondents would be able to provide this data. Comments: "Cannot measure" "Do not have the required systems" "Can provide but would have to ask another department" "Volume of data may be logged but we do not have the tools to extract and produce reports" 23

24 Number of subscribers to broadband and/or cable services by speed of connection? Eighty two per cent of respondents would be able to provide this data. Comments: "Don't have access to this data" Number of unsolicited (spam) s automatically blocked by your business? Do you automatically scan s for viruses, worms and similar malware? Sixty per cent of businesses would be able to provide this data. However, those that couldn't provided the following comments: "Don't have access to this data" "Data not available" Seventy three per cent of businesses would be able to provide this data. Those who answered no to this question either do not scan for viruses or required subscribers to pay extra for this service. Any other comments "My concern is that this is going to become a monthly technical questionnaire beyond a finance department's abilities. Taking into account the fact that computer personnel are not the best form fillers in the world, delays may be inevitable." "Your statistical reports (material) are still not user friendly - not easily accessible. Please look into this so that we can get the benefit of the data that we provide" 24

25 Annex E Details and Analysis of Users Feedback Additional data Users were asked if they required any additional data that could fall within the scope of the survey. The following requests were received: Business/ Household split Total monthly bandwidth (amount downloaded by users)* Number of spam blocked* Speed of broadband connection* Whether a business scans s for viruses* Publish actual number of subscribers * denotes question included on contributor's questionnaire. Periodicity Users were asked their opinion on whether the survey should be carried out on a monthly or quarterly basis. No objections were received to the survey being conducted on a quarterly basis providing the current level of timeliness was kept. Dial-up subscriptions They were also asked whether they still required the current breakdowns collected for dial-up connections. There was a mixed response to this question equally split between: Keeping the categories as they are; Having a metered/unmetered split; Combining all dial-up categories. Future of the survey As the proportion of broadband connection increases, the relevance of this survey may decrease. The total number of broadband connections is also collected and published by Ofcom. Users were asked whether the survey should continue after broadband reaches a dominant position e.g. 75 per cent of all connections. The general consensus was that the future of the survey should be reviewed again between 2006 and Particular attention would have to be made to providers of Local Loop Unbundling (LLU) services who may have a large impact on numbers and types of connections at that time - Ofcom may not be able to collect data from these businesses. Quality Measures Finally users were asked to 'score' the survey on a number of different categories. Scores were provided using the following scale: 1 = unacceptable, 2 = below expectations, 3 = satisfactory, 4 =good, 5 = very good. Category Average Score Comments Relevance 4.5 Accuracy 4.5 Comparability 3.5 Timeliness 4.5 Accessibility 2.5 "Difficulties finding data on the website" "Full back series not available" Coherence 4 Completeness 3 "Would like to see business/ household split" 25