TOOLS AND METHODS FOR MEASURING PUBLIC AWARENESS

Transcription

1 TOOLS AND METHODS FOR MEASURING PUBLIC AWARENESS Project: South-East European Digital Television Acronym: SEE Digi.TV Version A-1.0; Date: Page: 1/47

2 DOCUMENT HISTORY Version Status Date Author Comments Approved by A-1.0 Approved IVSZ, SINTESIO Approved version Project manager CONTENT 1 Introduction Planning a survey Defining research problem Designing the survey Qualitative versus Quantitative method Measurement methods Reliability and validity Sampling Probability sampling techniques Non-probability sampling techniques Sample size, weighting and error Constructing a survey instrument Types of questions Structured questions Unstructured questions Content of the questions Wording Order of the questions Interpreting survey results Concrete proposal for measuring public awareness in ASO process Introduction to the typical questionnaire Questionnaire for the ASO process Interpretation of results WEB based tool for measuring public awareness Page: 2/47

3 1 Introduction User acceptance is decisive factor in defining and executing a digitalization process and in particular switch-off period. Public awareness activities, which inform consumers/end users about the on-going digitalization and educate them about required actions, must be carefully planned and executed. The goal of executing public awareness activities is always to raise awareness of the consumers to the level which assures smooth and efficient Analogue Switch-Off (ASO) process. In order to evaluate success of such activities, one has to measure consumer s awareness. In order to be able to perform measurement, tools and methods needs to be developed and implemented regularly in each particular country. This document will provide an insight into planning and executing a survey for the purpose of executing ASO process, with a set of theoretical guidelines and very practical examples. Surveys are used to obtain information from the consumers in a structured way, which would be very difficult to get in other ways. Planning of efficient survey includes great deal of preparation, definition of the survey problem and objectives as well as definition of the survey design, while choosing the right methodology and research techniques. All this will be described in the first section. Many researchers think that it is an easy task to construct a survey but they are mistaken. It is a resource consuming and difficult task where many aspects must be considered like using selfadministered or other administered survey, if open-ended or closed questions should be used and in what order shall we ask the questions. From the point of view of the SEE Digi.TV project self-administered surveys are advisable to use where closed questions help the team to get useful responses to answer the original research question. The interpretation of the results should be planned already at the questionnaire construction phase since it influences which question types should be used. As the result of the data analysis charts, graphs, tables and pictures will be created that help the research team to visualise the final answer they got concerning the research question. Concrete proposal for measuring public awareness in ASO process in the form of a questionnaire is provided as well, in order to help partners being able to perform national surveys as efficient as possible. The construction of a good questionnaire is time consuming, with a lot of background Page: 3/47

4 research to be pre-conducted. Considering the status of ASO process in many SEE Digi.TV partner countries use of predetermined questionnaires can be of big help and directly executable through the proposed WEB based tool for measuring public awareness. On the basis of the theoretical background, methods, executable questionnaires and proposed tools for measuring public awareness provided in this document, SEE Digi.TV partners are able to periodically conduct national based public awareness surveys related to the Analogue to Digital Switchover process, which will help them to plan and execute the process efficiently. Page: 4/47

5 2 Planning a survey A survey in digitalization process is all about collecting information from households about the topic of digital television and their awareness, understanding and behaviours regarding the switchover to digital television. In order to achieve the main goal of such surveys, this is to provide feedback on the public awareness campaigns and activities, as well as to support a decision making process in ASO process, the survey planning shall be very systematic in order to have confidence in results. The research design is the overall research plan. It is providing the details of what are going to be done and how it is going to be done. Decisions about exactly how research is going to be carried out must be made before the actual start of surveying. It is important to prepare the explicit plan and leave nothing to be misinterpreted by the team conducting the research. This chapter will give somewhat more detailed view in theoretical elements of survey planning. 2.1 Defining research problem When a research study like this is to be undertaken, it shall be implied that the process: 1. is being executed within a set of approaches, such as qualitative or quantitative; 2. uses procedures, methods and techniques that have been tested in order to prove that those are valid and reliable, meaning that the correct procedures have been applied (validity) and that the research provides repeatability and accuracy of results (reliability); 3. is designed to be unbiased and objective, which means that each step has been taken in an unbiased manner and that conclusions are drawn without introducing anybody s own interest. If taking these criteria into respect, the process is called a research. On the other hand, one has to take into consideration that the degree to which these criteria are expected to be fulfilled varies from discipline to discipline. Research is described as a careful, systematic, patient study and investigation of some field, undertaken to establish facts or principles. Page: 5/47

6 Research is a structured enquiry that utilizes acceptable scientific methodology to solve problems and create new knowledge that is generally applicable. Scientific methods consist of systematic observation (or gathering data), classification and interpretation of data. Although such process is engaged in daily life, the difference between the casual day- to-day generalisation and the conclusions usually recognized as scientific method lies in the degree of formality, rigorousness, verifiability and general validity. In the ASO process, following factors are key drivers for defining a strategic approach for measuring public awareness and execute it in a structured way: ASO process is a complex process that can influence high number of population and could have negative implications if not executed accordingly; ASO process shall be supported with a public information campaign, a structured approach to inform population about digitalization, the quality of which shall be measured continuously; ASO process shall be based on a strong leadership and a reliable decision making process supported also by public awareness surveys. Defining a research problem in the ASO is all about recognising and understanding the problem of such process. The survey planners therefore need to be fully aware about the strategy and implementation plan related to digitalization in a particular country and understand many different challenges on this bumpy road. 2.2 Designing the survey Designing the survey is a next step in this planning process. Two basic types of surveys are used: questionnaires and interviews. Both questionnaires and interviews fall under the category of self-reports, whereas questionnaires are self-administered and interviews are administered by trained interviewers. This section is describing design issues that must be decided early in the planning process. The first rule is always to prepare a design which is reflecting the basic purpose of the research. The goal in measuring ASO public awareness is to record household s awareness about the process, their understanding and preparedness for the ASO. Understanding this, the survey planners Page: 6/47

7 need to determine the right set of questions, which shall be clear, precise and aligned with the content of a particular ASO information campaign. All questions need to prepared in a way, that can be answered by objective evidence. Designing the survey involves following steps: deciding which techniques to be used: qualitative or quantitative, choosing a measurement method, sampling strategy and administration approach, preparing the questionnaire developing a method for training of interviewers deciding about which background tools to be used for storing data and generating results determining set of analyses which are most appropriate to answer the original research questions and research goals The most important elements are further described hereunder. 2.3 Qualitative versus Quantitative method Two methods of data collection can be used: qualitative and quantitative. The difference lies in the approach and the types of questions asked to the responder. The quantitative methods are perceived to be more objective then qualitative methods, which are on the other hand more subjective. Qualitative methods are designed to provide the survey researcher with the perspective of target audience members through immersion in their culture or situation and through direct interaction with them. These methods help to answer questions such as how and why. The focus is to obtain information-rich data. Qualitative techniques include focus groups, field observation, in-depth interviews and case studies. Qualitative methods are often used as a preliminary step to help with planning research. Inductive analyses are most often used with qualitative methods, where data is collected first and then grouped into meaningful categories. The advantage of qualitative methods is in the fact that they can increase the depth of understanding, whereas they are flexible and allow the pursuit of new areas of inquiry. Page: 7/47

8 The disadvantages are that the data analysis and interpretation can be quite demanding and dependant on the researcher. There is also the problem that if qualitative research is not extremely well planned it can lead to nothing of value. Quantitative methods are adopted from the physical sciences that are designed to ensure higher objectivity and reliability. Using this method, maximum control over the questions and potential answers is sought and probability sampling methods to allow for statistical inference to the larger study population is often incorporated. The researcher is considered completely external to the actual research, and results are expected to be replicable, no matter who conducts the research. Quantitative methods help to answer typical questions such as who, how much, and how many. Where probability sampling is used, statistical analysis will provide precise estimates for study variables, such as frequencies, averages, ranges, means, and percentages, at a known and quantifiable degree of confidence. The intent is to gather data to test a pre-determined hypothesis and only answers to those questions/variables included in the questionnaire are collected. Questions are not open-ended and respondents are expected to provide short answers. This eases analysis, but limits the degree to which respondents participate and are able to provide explanations that they perceive. When using quantitative method, the survey planner identifies research questions as well as categories of answers during the design phase of the experiment. Designing a good quantitative survey requires a great deal of time and work. It takes a high degree of expertise to ensure that requested information is obtained. The main advantages of quantitative research are that it allows greater precision in reporting results and it permits the use of statistical analyses in order to ease the interpretation of results. The main disadvantage of using quantitative techniques is that some information may be lost as a result of predefined response categories. Quantitative techniques require careful expert planning to preserve information necessary to answer the research question. Although it is often appropriate to employ both quantitative and qualitative methods as they complement each other s strengths and weaknesses, in the ASO public awareness surveys, the Page: 8/47

9 predominant method to be used is quantitative. ASO process needs highly objective and precise information about the awareness in households. Especially when covering household awareness, acceptance and readiness for switchover, the survey shall come up with accurate, predetermined answers, which on the other hand mean that these surveys need careful planning. On the other hand, when analysing households perception and acceptance of the ASO process, the survey can use qualitative methods as well. 2.4 Measurement methods The most important activity when designing quantitative approach to the survey is to determine how variables will be measured. Measurement is referred as assigning numerals to objects, events, or properties of those objects or events according to certain rules. A numeral is a symbol that has no value until a level of measurement is determined, whereas rules specify the way numerals are to be assigned. Terms that may be used to describe certain types of measures are following: Input measures are quantifying variables which are thought to be causes for particular outcomes; Process measures are quantifying processes that cause changes in outcomes; Output measures are quantifying objects that are the result of some input or process; Outcome measures are quantifying events or states that result from some input or process; Variables are conditions or events that vary over time or across different situations, while having more than one value. Variables can be measured empirically, that is, they can be measured directly; Constructs are abstract. They are formulated to serve as descriptive explanations and are not directly measurable. Constructs shall be defined according to the particular research topic and environment. There are different ways how to perform measurement, usually referred to as levels of measurement. The meaning of the numerals assigned to classes of objects or events depend on the level of measurement used. Page: 9/47

10 Nominal scale is the level of measurement where the numerals assigned simply classify things. In other words, the numerals are labels that stand for a particular category without any particular mathematical significance. An example of nominal measurement is postcode. Ordinal scale is level of measurement where numerals convey more information than do nominal scales. Objects measured at the ordinal level are ranked along some dimension, usually from smaller to greater. These measures give information about the relative position of the properties being measured and have no information about the difference between them. Interval scale is the level of measurement where a score on an interval scale indicates rank order just as an ordinal scale does, but in addition it incorporates the property of equal differences between scores. An example of an interval scale is the age of the respondent. Interval scales can be processed with arithmetic operations, for example, it makes sense to talk about average age. Ratio scale is the highest level of measurement, which has all the properties of the other scales; that is, the scores categorize, they use rank order, they have equal intervals, and in addition, they have an absolute zero. Because ratio scales have the property of absolute zero, proportional statements can be made with all mathematical operations possible on measured scores. Ratio scales are usually used in hard science, but when it comes to social science or in survey research (such as measurement of public awareness in ASO processes) they are not so useful. In this case, the survey is often focused in attitudes, opinions and knowledge people hold rather than to externally measurable events. For these cases, the researcher must consider the appropriate type of statistical analyses and must ensure that the appropriate level of measurement is used. For most rating scales used in surveys, an interval level of measurement is assumed. Ratings Scales Rating scales are the most commonly used, whereas there are several basic types of ratings scales: Numerical scales are providing a respondent with a sequence of defined numerical or alternative (such as, Strongly Agree, Agree, Disagree, Strongly Disagree) choices. Page: 10/47

11 Forced-choice scales are providing two equally desirable or two equally undesirable choices and the respondent should then choose between these two. Graphic scales present a straight line vertically or horizontally with two opposite choices on either end of the line. Respondents mark the line at some point indicating the degree to which they rate the item relative to the two extreme positions. 2.5 Reliability and validity The execution of survey without proper testing can result in poor quality or even wrong results. One of the reasons for testing is also the fact that it is very difficult to predict how respondents will react to the survey questions or ratings scales without prior testing. It is highly recommended to execute at least one pilot study to ensure its reliability and validity. A pilot study can be also a scaled-down version of the survey primarily conducted to test the survey instrument itself. Although the survey instrument presented in this document is a result of in-depth planning, analysis and best practice study, it is highly recommended that the respective institution conducting a survey in the ASO process conducts pre-testing and gets more familiar with the proposed survey instrument. This includes also detailed analysis of proposed questions and their amendments in case this is needed. Through the execution of pilot testing, the researcher shall strive to assure reliability and validity: Reliability is assured if the instrument consistently gives the same answer. Any measurement instrument must be reliable to be useful. This can be assured through following components: o Stability means consistency over time and can be referred to as test-retest reliability. This means that a stable instrument will give the same result tomorrow that it gave yesterday, whereas one shall take into consideration that people perspective can change over time. o Consistency refers to the fact that different parts of a scale should be consistent in the measurements they yield. o Equivalency which means that two equivalent forms of a test should return the same measurements, when assessed by researchers/observers. Validity is assured if the instrument is measuring what it is designed to measure. A valid measure is an accurate, useful measure. There are several types of validity, which need Page: 11/47

12 to be taken into respect, namely Face validity, Predictive validity, Concurrent validity and Construct validity. 2.6 Sampling When every member of the survey interested population is questioned, the survey is called a census. Conducting a complete census is usually cost and time prohibitive, therefore surveys use samples to be execute on. Sampling occurs when a subset of the population (or other unit) under study is selected from the larger group (the entire population under study). By studying the findings from that sample it is hoped that valid conclusions can be drawn about the larger population from which the sample was taken. Since a sample is a subset of the population, one of the most critical elements of survey planning is designing an appropriate sampling procedure. Using an inappropriate sampling procedure can render results invalid because results from the sample survey cannot be extended to the entire population. In order to be able to generalize results obtained on a sample to the whole population, the sample should be representative. The sample is representative when the subset of the population whose characteristics correspond to those of the population as a whole. Obtaining a representative sample depends on starting with the appropriate sampling frame. The sampling frame is the list from which the sample is selected. Random-digit dialling computer programs are sometimes used to obtain survey samples. This technique is convenient but excludes members of the population who don t have a telephone. In addition to choosing the appropriate sampling frame, the researcher must select an appropriate sampling technique. Sampling techniques can be divided into probability sampling and non-probability sampling. Within each of these a variety of subcategories exist and a number of ways of selecting the sample can be used. Both probability and non-probability sampling methods seek to gather data that provide a fair representation of the larger population, although the definition of representative varies between the 2 methods. Probability sampling techniques rely on statistical theory as a basis for extrapolating findings from the sample population (n) to the larger study population (N). By contrast, non-probability Page: 12/47

13 sampling does not utilise statistical theory to support inference from a sample population (n) to the study population (N), but rather relies on a more subjective determination of the degree to which a sample represents the larger study population. The choice of which technique to follow depends on the intended use of the information and the importance placed on objective (probability sampling) versus subjective (non-probability sampling) determination of how representative the sample is Probability sampling techniques Probability sampling allows for statistical inference. Statistical inference makes use of information from a sample to draw conclusions (inferences) about the population from which the sample was taken. The estimates are representative of a larger population, from which the sample population is taken, at a known and quantifiable level of confidence or probability. Estimates are given in ranges, called confidence intervals, although they are often expressed as a point estimate +/- a number of percentage points. Probability sampling is almost exclusively used with quantitative data collection methods. The essence of probability sampling is that each unit of study (e.g. household) in the study population for which the estimate is desired must have an approximately equal probability for selection and inclusion in the sample. In order to ensure that this critical criterion is met, an exhaustive sampling frame must exist or be created for the unit under study (households). A sampling frame is a complete list of all the potential units of study in the population from which the sample will be taken. In many countries, it is impossible to find an existing sampling frame at the unit of study level and it is too costly to construct one. In these cases, cluster sampling is used. Cluster sampling aggregates the unit of study into groups or clusters for which a complete or nearly complete list is available. Although cluster sampling is very commonly used, it is rarely employed appropriately. Expert guidance should be sought in applying cluster sampling and determining the appropriate number and proportional weighting of clusters. Determining the appropriate sample size is based on a set of parameters concerning the degree of confidence desired in the estimate, the design effect of the sample, the degree of tolerable error and the proportion or mean estimates for the variable of interest. Expert guidance should be Page: 13/47

14 sought in determining the appropriate sample size needed if probability sampling is being used. Several probability sampling types are used: Simple random sampling is a process in which each member of the population has an equal chance of being selected. Individuals are selected from the population by some purely random method such as drawing names from a hat or selection from a random number table. Systematic random sampling involves selecting every n th member of the population from the sampling frame. Systematic random sampling may be appropriate if the list being used as the sampling frame has no regular pattern. A stratified sampling technique is used when adequate representation of certain subsets of the population must be ensured. Proportionate stratified sampling involves randomly selecting individuals from desired segments of the population. The number selected from each segment is based on that segment s proportion in the population. Cluster sampling is a technique in which the population is divided into clusters, sometimes geographical areas. Each cluster or area is sampled using simple random sampling. Multistage sampling involves first dividing the population into clusters then randomly selecting clusters or geographical areas to be sampled. From the selected clusters, individuals are chosen using simple random sampling. For the purpose of measuring public awareness in ASO process, only probability sampling techniques shall be used, possibly with a combination of sampling types as described above. Most commonly, the Cluster sampling technique with a division into geographical areas is used for measuring public awareness in ASO Non-probability sampling techniques Non-probability sampling also seeks to draw conclusions about the larger population under study through using a selected sample or subset of that population. Non-probability sampling is almost always used for qualitative data collection methods and is not advisable for samples intended to measure public attitudes, opinions, or general awareness. Non-probability sampling is sometimes used for special purposes such as pilot studies (testing the instrument itself) or marketing studies in which some particular target segment of population is of interest. Page: 14/47

15 Following non-probability sampling types can be used: Availability sampling is sampling from a collection of readily accessible subjects (such as a group of shoppers in a mall). A volunteer sample is composed of people volunteers, which have particular characteristics that distinguish them from the general population thus, the results of volunteer surveys do not generalize well. Purposive sample includes people who are selected on the basis of specific characteristics or qualities. Quota sample includes people who are selected to meet a predetermined percentage because they have certain characteristics Sample size, weighting and error In order to be able to generalize the results acquired on the basis of a sample, the sample itself must be large enough to represent the population under survey. In general, there are no rules for determining an appropriate sample size; however a larger sample is of course more representative. On the other hand, a large sample costs more. It is therefore advisable to take a compromise between an appropriate sample which is large enough to represent the population, but small enough to be cost-effective. An important consideration in determining sample size can also be the estimated frequency of the phenomenon to be measured. If something fairly frequently is to be measured, a smaller sample will probably be enough to measure it, and one the other hand, if something relatively rare is measured, a larger sample is probably needed. Sometimes it might happen that after the survey is complete and the data analysis has begun, a researcher discovers that certain segments of the population are underrepresented (such as people living in the rural areas or elderly people). Rather than re-sampling or re-surveying, a statistical technique called sample weighting can be used to overcome this discrepancy and correct the results of the survey. It shall be especially noted that sample weighting is not a substitute for using an appropriate sampling design, but a corrective measure that can be used when samples are not representative enough due to uncontrollable factors such as low response rates among certain segments of the population. Page: 15/47

16 Estimates calculated from a sample may differ from those that would be obtained from a full census of the population. This difference is called sampling error. The margin of error can be calculated to indicate the precision of an estimate based on a sample. The sampling error is calculated in per cents, meaning that the sample estimate and the value that would be obtained from a full census will differ by the margin of error or less. Page: 16/47

17 3 Constructing a survey instrument Questionnaires are used to collect data on subjects demographics, personal histories, knowledge, behaviours, and attitudes. Constructing a survey is a special activity that influences and partially already determinates the potential results already at the very beginning of the research. To construct a survey is a more challenging task than many researchers believe. Purely designed questionnaires will bring unsatisfactory data and results only the loss of many resources. The wording, the question types, the structure of the questionnaire are all important details that must be carefully planned in order to have a reliable survey instrument at the end. Most of the rules that must be used when creating a survey are based on common sense however it is important to have these rules mentioned since it is easy to make failures that bring the survey into a wrong direction and will serve the researcher with useless and not reliable results. It is well known that self-administrated surveys (where the respondent fills in alone the questionnaire) are less expensive than interviews, they can be distributed via mail or but they will provide more unsatisfactory, incomplete responses since the respondent cannot ask anybody when a question is not clear enough. The present document will give an overview on how to construct a self-administrated survey. When creating a survey the researcher must go from step to step on the following list: 1. State the problem/define your research aims: describe the need why you intend to run a survey 2. Plan the project: plan resources (human and financial), timeline, your target groups/population and the (representative) sample (it might be obvious or not so obvious depending on the survey membership of an association or the population of a city), dissemination channels etc. Count with 20% response rate. Population: all the members of the group you are interested in. Sample: the subset of the population selected to receive the questionnaire Page: 17/47

18 Respondents: the subset of the sample that actually complete and return the questionnaire 3. State the research question: what is the pertinent research question? Formulate clear, answerable, interesting and important questions 4. Review the literature: get familiar with the topic, check if there are similar surveys or the question is already answered. If there are similar surveys you might consider the adaptation (you have to ask for permission) or you can use their results for comparison. The adaptation might minimise the required resources. 5. Develop/adapt survey items: developing good, appropriate and useful questions. The item is composed of two parts the question or statement and the response (if pre-defined response options are used). Avoid the temptation to add few more questions only because you are constructing a survey. 6. Construct the survey: Use the KISS rule: keep it short and simple! As soon as all items (questions/statements and responses) are composed the whole survey can be constructed. The order of the questions and the order of the possible answers may affect the respondent therefore both must be considered well. Grouping of questions and instructions are the most important parts to consider at this stage. Use always easy questions at the beginning that helps the respondent to roll with the questions. Difficult and sensitive questions should be placed around the end of the survey. 7. Pilot test the draft survey: the poorly written and redundant questions can be identified by doing pilot tests with colleagues or with candidates from the target group. It can be identified also if a question is misleading or confusing the respondent. As part of the pilot test the responses should be analysed too. This analysis will help us to see if the questions are well defined from the analysis s point of view. 8. Administer the survey: the introduction (cover letter or pre-notification letter) helps to convince the respondent to fill in the questionnaire ordinary. It should inform who is collecting the answers, on the objectives of the survey, what the expected outputs are and why the person s responses are important for the study. It is proved that these letters can significantly increase trust in the survey and can convince the respondents to take part in and to dedicate time for the survey. This section will provide an overview on the types of questions you might use as well as will give useful tips regarding the content of the questions. Page: 18/47

19 3.1 Types of questions There are different types of questions but they can be grouped into two bigger groups: structured (closed choice of alternative replies) and unstructured (open-ended free text answers) questions Structured questions It is more typical to use structured questions in case of surveys and to use unstructured ones in interviews but it is not mandatory or compulsory. The researcher who sets the questions has always the freedom of using both types of questions but it must be always seriously considered which ones might bring us the wished results. Probably we might state that it is more difficult to write structured questions but the results are easier analysable. Advantages of structured questions Easy to code(example: 1 = Strongly Agree; 2 = Agree; 3 = Disagree; 4 = Strongly Disagree) Easy to enter Easy to analyze Easy to present Quick turnaround Enhanced reliability Less researcher bias High degree of anonymity Disadvantages: Harder to develop questions and response categories May force invalid responses Less depth and substance Respondents unable to explain, qualify, or clarify answer Page: 19/47

20 Dichotomous questions The name of this question type already refers on the characteristics of these questions. The responder can choose in case of dichotomous questions from two possible answers e.g. yes or no (other examples: true or false, agree or disagree). Have you already heard about digital switchover? Yes No Are you aware of the set top box subsidies? Yes No Questions based on the level of measurement We might measure different topics for instance with a nominal scale. The occupation or for instance the educational level might be measured with this method. We must note that the numbers have no meaning in this case e.g. the university degree is not twice as worth as the high school degree. Page: 20/47

21 0 grammar school 1 High school degree 2 university degree Ranking of questions is also often used in surveys when the researcher would like to learn more on the order of preference of the respondent. What solution do you prefer for? Please rank you preference from best to worst where 1 is the best and 5 is the worst. Provide a list of 5 possible answers. Please note that this question type might cause confusion since the respondent might choose as first that option he have heard about the most and not his real preference. Interval level questions are also widely used in surveys and are called often as Likert response scale or bipolar scale due to the fact that the two ends of the scale are opposites and there is also a neutral point in the middle. The scale contains usually 5, 7 or 9 rates. The digital switchover process was necessary. Strongly disagree Disagree Neutral Agree Strongly agree A very similar method is when you are assessing an object on a bipolar adjective pairs (5 point rating scale with opposite adjectives. Unnecessary Simple Cheap Please state your opinion on the digital switchover process Very much some-what neither some-what very much necessary complex expensive The interval measures last type is the so called cumulative or Guttmann scale. The researched in this case asks for your agreement in case of several statements. Usually when you agree with one of them you will agree with all. Page: 21/47

22 Pictorial scale In case of this type the possible answers of the respondents can be smiling faces where the respondent can mark how s/he feels about the answer Visual analogue scale By using visual analogue scales the respondent can mark on a usually 10 cm long line his/her opinion regarding a given topic. The two ends of the lines are opposites like agree and disagree Filter or contingency questions Depending on the topic of your survey you might need to filter the respondents if they are enough qualified to proceed with the subsequent questions therefore you are asking will use a filter or contingency question. This type of question is useful if you want to ask different questions from the respondent if s/he has experience or some knowledge in the given topic or if s/he has not. Do you have already access to digital stations?? Yes No Depending on the answer you will proceed with different questions. E.g. you might ask the opinion of the respondent on the quality of the service or if s/he is using pay-service as well if s/he answered yes, but there is no reason to proceed with these questions if the answer was No. As it is suspectable filter questions can make our survey more complex and it might be necessary often to use several level of filtering questions in order to direct your respondents to the correct subsequent questions. It is advisable that no more than 3 levels of filter questions are used. Too many jumps might have a negative effect on the respondent s willingness to answer the survey. Another advice is to use graphical help when you have two levels (e.g. arrows with boxes, jump to a new page rather than to a given question) Unstructured questions Unstructured or open-ended questions are used more in case of interviews (however not exclusively) where the respondent can freely answer the question with his/her own words in the Page: 22/47

23 length s/he prefers. Since there are no pre-defined answers in this case the respondent has to think on his answer. However this might bring more detailed answers in some cases it might result also low level or useless responses. The analysis of open-ended question requires more resources since it is not so easy code decode but the task can be facilitated with text mining software. However the open-ended questions provide different and more colorful results (if the respondent answers it well) the number of such questions should be kept minimal in selfadministered surveys. If it is necessary to use open-ended questions then they should be placed around the end of the questionnaire. 3.2 Content of the questions The content of the question is the most critical point. If the questions do not address well the content that the survey should get at than the desired responses will not be reached. Therefore we strongly advice to ask the following questions from yourself when writing the questions: Is the question necessary/useful? You must investigate if the question will provide you the necessary information and if it is in the right place. One question or it should be split? It is a common mistake that two questions are mixed in within one question. The easiest way to find these wrong questions if there is an and or an or in the question. E.g. Do you believe that the digital switchover was a necessary and well communicated process? Another typical case where it is useful to split one question into two is when you would like to know for instance if the respondent is in favour of switching to digital television. In this case it might be interesting to know the opinion of the other persons living in the same household. The question might not provide enough detailed context or it does not specify the intensity of the respondents attitude or belief. These are all important details that might result that further questions will be necessary. Page: 23/47

24 For example, if they say they support the digital TV, you probably should also ask them whether they ever watch it or if they would be willing to have their tax dollars spent on it. It's one thing for a respondent to tell you they support something. But the intensity of that response is greater if they are willing to back their sentiment of support with their behavior. Do respondents possess of the requested information? When writing the questions it must be always considered that it has no point to ask a question for which the respondent might not have the necessary knowledge to answer. Therefore if the question is important to be asked but a given percentage of the respondents might have no knowledge to answer it is advisable to ask a filter question. Is the question enough specific or too general? Sometimes the question is written in a way that it will provide too general information. E.g. How much do you agree with the digital switchover process? The answer agree or not agree does not explain the real attitude therefore the question must be other formulated. Is the question sufficiently general? Not only the too general but also the too specific questions are not useful since they may hinder the respondent in giving the correct answer s/he wants to give. Is the answer influenced by the writer of the question? The person who writes the question might influence the future answers by formulating the questions in different ways. If you are in favor of the digital switchover process therefore you ask the following question: What do you see as the benefit of the digital switchover process? In this case you are asking only one side of the issue. In this case you can get a full picture only if you are asking also the other side of the issue otherwise the answer is not balanced. Will the respondent answer truthfully? There are questions that might the respondent not willing to answer because of several personal reasons. Such questions might be the age, religion, earnings etc. If our research needs such Page: 24/47

25 information than we can overcome this problem by giving response brackets to choose from (20-30 years, years etc) or by using a hypothetical projective respondent: people you know. Allow the option Don t know or Not applicable Allow the option Don t know or Not applicable in all cases where you are not certain that all respondents will be able to answer the question. (Similarly offer the option None or Other if you are providing a list of possible answers) Do not use abbreviations or technical terms A very typical mistake that researcher make in their survey is that they use special terminology or abbreviations that are obvious for them but not so obvious and well known by the respondents. Such mistakes can easily result that the respondent gets annoyed and stops to fill in the survey. Always indicate if multiple choices are possible The respondents feel often that more than one of the pre-defined responses meets her/his answer therefore if possible it is always necessary to make a remark that multiple choices are possible. 3.3 Wording As it was already mentioned before the wording of the question is one of the most critical parts of constructing a survey. Different wording might cause confusion or might lead to a different result. The following questions might help you if your survey questions are appropriate: Is it possible to misunderstand the question? Many times the question can be understood in different ways and the interpretation of the responder will decide the meaning and the potential answers. (This is especially critical in case of unstructured questions, where the answers are not predefined.) Does the question make any assumptions? It has been already mentioned that the question might assume that the respondent knows the term that is used or is aware of the information or possesses on the knowledge that is needed in Page: 25/47

26 order to answer the question. In case of such questions a filtering question might solve the problem. Is the timeframe specified? The use of the following words is usually bound with time frame: will, could, might, may. It must be ensured always that if the above words are bound with a timeframe than the question contains this information. E.g. Do you think the A/D switchover will be used by 50% of the population in Germany? Personal-impersonal wording It is possible to switch a question from impersonal to personal by changing few words. The question will refer to the same topic but the level will change. Is the question too direct? Too direct questions might lead to an unsatisfactory result, since it might be threatening or disturbing for the respondents, especially if the topic is bound with a negative memory. Try to avoid modifiers, like almost everyone, usually etc. State a positive statement not a negative 3.4 Order of the questions The placement of the different questions is essential. The sensitive and the most important questions placement must be well planned. At the beginning some warm-up questions should be made that helps the respondent to get familiar the survey and start rolling. The key questions should be not placed at the end because the respondent might get bored of the survey and stops filling it in, while if it is too early in the row the respondent might not be ready for the key questions. If the survey should include sensitive questions than they must be placed in the way that the respondent gets ready for answering them. Page: 26/47

27 It is often helpful to have a transition sentence between sections of your instrument to give the respondent some idea of the kinds of questions that are coming. Checklist of Considerations There are lots of conventions or rules-of-thumb in the survey design business. Here's a checklist of some of the most important items. You can use this checklist to review your instrument: start with easy, nonthreatening questions put more difficult, threatening questions near end never start a mail survey with an open-ended question for historical demographics, follow chronological order ask about one topic at a time when switching topics, use a transition reduce response set (the tendency of respondent to just keep checking the same response) for filter or contingency questions, make a flowchart Page: 27/47

28 4 Interpreting survey results The general objective of each survey is to come to some conclusions in respect of the basic research question that have been raised at the beginning of the research. The analysis of the data must be planned already in the questionnaire construction phase. The pilot test of the survey is important from the questions point of view but it has significant importance for the analysis too. This is the first feed-back that the researcher might have if the data that the survey provides is useable or not. There are some questions that you should ask from yourself before selecting the analysis method: Are you looking for relationships among variables? Are you interested in comparing groups of respondents? Are you interested in looking at changes over time? The easiest and commonly used method for analysing data is to import responses data of the electronic survey into data analysing software that help to create charts and provide interpretable results. Such tools are Excel, Access, SPSS etc. Many times the survey tool (online) also supports the analysis phase with built in programmes. In these big databases the lines are the single responses while each column represents a specific variable, i.e. the data for that variable for all respondents. Before starting the analysis it is always advisable to check the data for errors. Check if any errors occurred when entering data or if the respondent provided inconsistent answers. For ease of data handling and analysis the values that variables can take are usually designated by numeric codes, even when the variable is a nominal one. The response rate can be calculated by dividing the number of people who submitted a completed survey (80% or more of questions answered) by the number of people you contacted or attempted to contact to complete the survey. If the response rate is below 70 percent, determine if the sample is representative of the target population by comparing sample and target population means for characteristics that would likely affect responses, such as race, age, grade point average. In case the representativeness of the result might be questionable you can collect additional questionnaires that increase the response rate above 80%. Another option that might be used in some cases is to weight results so that the attitudes of important subgroups are not underrepresented. Page: 28/47

29 Frequencies for each response option can be calculated by counting the number of respondents who selected each response choice and divide these frequencies by the total number of responses to the question to compute percentages. Cross-tabulations/correlations are used to see relationships between responses for two survey questions. Cross-tabulations can highlight contrasts between groups of participants or findings that are consistent across groups. Analyses of differences are used to test whether the difference between scores of two groups or two different conditions of testing is statistically significant. There are many further different statistics and test that can help to show differences or correlation between variables, like Chi-Square, Analisis of variance etc. In many cases the online survey tool supports the analysis phase too. It helps to create statistics, graphs, tables and different visualisation tools. It is very important that the results of the analysis is visualised with different tools that help the final user of the research results to get a quick or even a detailed overview. The following visualisation tools are usually used: Tables are used to show many options and summarize data Pie chart: to show the proportion of each option from the whole. It must have a title that describes the chart and a key that helps understanding and interpreting the chart. Bar graphs: it is primarily used to compare groups or categories of scores. It must have a title that describes the graph and a key that helps understanding and interpreting the graph. Histograms are a special type of bar graph for displaying frequencies of occurrence. Line graphs: should be used to show changes over time All visualisation tools can help to interpret data but they can be misleading too therefore they must be handled careful. Keep in mind that the purpose of a graphic is to increase understanding Page: 29/47

30 of the procedure you used or the results of your survey. Design graphics to emphasize the information you wish to convey. Page: 30/47

31 5 Concrete proposal for measuring public awareness in ASO process 5.1 Introduction to the typical questionnaire The overall objective of measuring public awareness in ASO process is to: check the awareness and understanding of public/households about activities related to ASO check the readiness of public/households to switch-over from analogue to digital reception provide high quality data that will be used to help plan and execute public awareness activities and increase awareness and understanding about ASO ensure a smooth transition to digital for all households The typical questionnaire prepared for measuring public awareness in ASO process can be executed on the basis of three key measures: 1. Awareness of ASO (awareness); this group of key measures explores awareness of the national Switchover plan s to switch-over to digital television and the analogue switch-off timeframe; 2. Understanding of how to get ready for digital television (understanding); this group of key measures explores a household understanding how to convert to digital television; 3. Conversion of a household, to be able to receive digital television (conversion); This group of key measures explores the conversion of households to digital television including the method of conversion (for those that have converted). For additional information and in-depth surveys which would help national authorities responsible for ASO process to understand and check the outcome of the ASO process, following key measures could be added: 4. Attitude to the digital switchover (attitude); this section explores households intention to convert to digital television as well as their attitude to the digital switchover. This part of survey is exploring households positive or negative attitude as well as the main reasons for such an attitude; Page: 31/47

32 5. Satisfaction with digital television conversion (satisfaction); with this key measure, information about satisfaction with the conversion process and digital television as a potential new service is gathered. The typical survey for ASO process is based on asking questions to the responders for two key measures awareness and understanding, whereas it is flexible for its use in national based ASO plans as well as regionally based ASO plans. Particular care shall be taken to ensure the data is accurate, reliable and timely and that the methodology and assumptions are robust. The structure of the typical questionnaire for ASO process is as follows: Section 1: Introduction Section 2: Awareness of switch-over Section 3: Understanding switch-over Section 4: Conversion to digital TV Section 5: Household and respondent characteristics It is highly recommended to test the survey procedure and a questionnaire prior to execution and amend it accordingly. 5.2 Questionnaire for the ASO process SECTION 1: Introduction Note: The person executing the survey shall have prepared an introduction explaining following: Introduction of the institution and person conducting the survey Background and objectives of this survey Time needed to conduct the questionnaire Confidentiality 1. Can you confirm that I m speaking to the person aged 18 years and over, who knows the most about the TV equipment in your household? Page: 32/47

33 Note: If NO, then ask if it is possible to speak with the right respondent and repeat the introduction. 1 Yes 2 No 3 Don t know SECTION 2: Awareness of switch-over 2. Do you watch any TV set in your household with the help of rooftop or indoor antenna? 1 Yes 2 No 3 Don t know 4 Don t have TV set Note: Continue with the questionnaire with all except those who answered Don t have TV set. 3. Have you heard or read anything about the fact that name_of_country will switch-over from analogue to digital TV (or signal) // or // switch-over to digital TV (or signal) by turning off the analogue signal? Note: This question could be amended in order to be more aligned with the information campaign or specifities of the country s ASO process. 1 Yes 2 No 4. Do you know when approximately will switch-over happen in your region/area? When? Page: 33/47

34 Note: This question shall be amended if only national approach is foreseen (delete in your region/area ). The respondent shall provide the answer about when the switch-over will happen. Yes is valid if at least information about Year and Month is correct. 1 Yes (provide answer) 2 No SECTION 3: Understanding switch-over 5. Do you know what should those who receive signal with rooftop or indoor antenna do in order to switch-over? 1 Yes 2 No Note: If the answer to this question is No then skip the next question. 6. Can you explain what? Note: Multiple responses are possible but interviewer shall not read possible answers. Possible answers could be amended in order to be more aligned with the information campaign or specifities of the country s ASO process. If none of answers match possible answers below or the respondent answers with Don t know, then the reviewer shall change previous answer to No. 1 Buy a STB Buy a new/appropriate 2 TV set Arrange reception 3 antenna Other (provide 4 answer) 5 Don t know Page: 34/47

35 7. Do you know where all of those interested in this can get more information? 1 Yes 2 No Note: If the answer to this question is No, skip the next question. 8. Where? Note: Multiple responses are possible but interviewer shall not read possible answers. Possible answers could be amended in order to be more aligned with the information campaign or specifities of the country s ASO process. If none of answers match possible answers below or the respondent answers with Don t know, then the reviewer shall change previous answer to No. Published telephone 1 number/call centre 2 WEB page 3 Brochure/leaflet 4 Newspaper/TV Other (provide 5 answer) 6 Don t know SECTION 4: Conversion to digital TV Note: This section questions (no. 9 and 10) shall be asked only if answer to the question no. 2 is Yes, meaning that the household is receiving the signal over the air. 9. Have you already converted to digital TV, so your household is able to receive the signal? 1 Yes Page: 35/47

36 2 No 3 Don t know Note: If the answer to this question is No or Don t know, skip the next question. 10. How did you convert? Bought a new TV set 1 with digital tuner Bought and installed a 2 STB 3 Other (specify) SECTION 5: Household and respondent characteristics Note: Interviewer shall inform the correspondent that following questions are about him/her and his/her household. After that, information about sex of the respondent shall be recorded: 11. Record sex 1 Male 2 Female 12. What is your current age? Note: Interviewer to decide which group does the answer belong to Over 60 5 Refused Page: 36/47

37 13. What is the highest educational qualification you have completed? Note: The answer groups shall be aligned with national classification of educational qualification. Didn t complete 1 primary school 2 Primary school Vocational school 3 (lower level of secondary school) Highest level of 4 secondary school 5 University diploma Master diploma or 6 PhD 7 Other (specify) 8 Refused 14. Where do you live? Note: The exact classification shall be determined prior to conducting surveys. Countryside or village 1 (rural area) Town or smaller city in 2 the rural area 2 City in the urban area Bigger city or 3 metropolitan area (urban area) 15. What is the postcode where you live? Page: 37/47

38 Note: This question is used to determine and double check the area of household in case survey and ASO is based on regional approach 1 Provide answer 16. Where are you employed? 1 Industrial sector Non-industrial sector 2 (public) 3 Self-employed 4 Farmer 5 Housewife 6 Retired 7 Student 8 Unemployed 9 Other (specify) 5.3 Interpretation of results If survey such as this one is conducted, the interpretation of results is highly important for the execution of ASO, implementation of public awareness and many other stakeholders, who are directly or indirectly involved in ASO process. The quality interpretation of results may contain tables, graphs, pie charts and associated statistics, with a descriptive prose, providing a structural analysis of the survey results. Once the data has been entered into the statistical package, the analyses required to answer the research questions can be performed. Analysing the survey results is done in order to answer the original questions that were posed for the evaluation. It allows drawing conclusions which allow interested parties to get feedback, steer the ASO and execute decision making process. Following shall be taken into consideration when executing analyses and interpreting the results: Page: 38/47

39 If regional ASO approach is taken, the representation of results shall be strictly regionally driven, thus providing the same analysis for every period. One can even provide a comparative regional analysis, which can provide valuable feedback on the quality of communication activities region by region. More complex analyses such as this one is proposed when comparisons are needed between subgroups of the population or for measurements taken at different times. Statistical analysis aims to show that results are not just due to chance or the luck of the draw, but the result of activities performed on the field. For this purpose, the survey shall be conducted periodically, whereas the results shall be interpreted on the basis of periodical comparison. Statistical expertise shall be combined with stakeholder interpretation. Even though the results may be statistically significant the differences seen may not be very meaningful in terms of decisions that need to be made. Results should not only be interpreted through statistical tests but also through discussion with stakeholders as to what the results might mean. Although proposed questionnaire is a flexible one and can always be aligned with national specifities, it is proposed to use simple descriptive analysis in order to avoid getting tide up in detailed analysis that may not help to answer elemental research questions KEY MEASURE 1: Number of households (HH) with terrestrial reception: With this key measure, the share of population, who will be influenced and concerned by the ASO is provided. On the other hand, this measure also provides the share of population on the terrestrial platform. KEY MEASURE 2: Number of HH who are aware about ASO: This key measure will give us the answer to the question about how well the households are informed about ASO. This particular measure is giving us the share of households who are aware that ASO will happen in the future. KEY MEASURE 3: Number of HH who are aware about ASO date: This key measure will provide a feedback on the quality of communication and acceptance of communicated information about when the switch-off will happen. In order to be as focused as Page: 39/47

40 possible, the correct answer provided by the respondent is considered if the correct Year and Month is provided. KEY MEASURE 4: Number of HH who understand how to prepare for ASO This key measure is providing the share of households who understand how to prepare for ASO. Moreover, this is confirmed by the secondary question, where respondent specifies what exactly needs to be done in order to confirm that the understanding is valid. Further analysis about provided answers will give a feedback on what exactly do households understand by preparing for ASO and how well were details about getting ready for ASO communicated to the population. KEY MEASURE 5: Number of HH who know where to get information about ASO This key measure is providing a share of households who know where to get more information about ASO. Moreover, this is confirmed by the secondary question, where respondent answers about which communication channels can be used for getting further information. Further analysis about provided answers will give a feedback on which possibilities to get more information were appropriately communicated to the population. KEY MEASURE 6: Number of HH who already converted to digital TV The last two key measures provide information about how many households already converted to Digital TV. This particular key measure shall be analysed periodically and is increasingly important when the ASO is approaching in order to determine the extent of problems when switching off the analogue TV. The share of households who already converted shall be measured only with the relevant households who are receiving the signal over the air. KEY MEASURE 7: The method of conversion to digital TV This key measure provides the share of households who converted by buying new/appropriate TV set with integrated receiver and those who bought and installed the STB. Structural analysis: The typical questionnaire for measuring public awareness in ASO process includes also the section with household and respondent characteristics such as: Page: 40/47

41 sex age educational qualification type of settlement (urban/rural) region employment Additional structural analysis shall be performed in order to present interesting results, while combining key measures presented above and household/respondent characteristics. The researcher is proposed to include a structural analysis of a particular key measure under that particular section in the form of descriptive prose, supported by exact figures (percentages). Page: 41/47

42 6 WEB based tool for measuring public awareness The web survey as a whole most closely resembles the type of surveys which were traditionally conducted by sending out mails or using web sites. In this case, researcher is using an to send a link that takes responder to a HTML form, which is designed for completion and submission through the computer. Information is collected quickly, because as soon as the respondent has finished taking the survey, its responses are immediately submitted. Web surveys can also be embedded on researcher web site using what is known as an IFrame. The researcher can also create a Pop-Up survey on the web site so that when a site visitor visits a page the HTML form pops-up from the web site. If they have pop-up blockers enabled then this method will prevent them from taking the survey so researcher can also create a link to the survey which can be display on designated web site. The web survey is a cost effective way of administering a survey that allows researcher to collect large amounts of information without having to pay for interviewers, paper supplies or postage, and does not require separate data entry for responses to be processed. The rise in web based surveys is due in no small measure to the increasingly widespread availability of computers. Particularly in organizational or professional settings, the ability to receive a questionnaire and complete it at home or in the office on a computer is very convenient for most people. The researcher can expect to wait at least a few weeks for a questionnaire that is mailed out to a respondent to be returned. A web survey allows rapid collection of data in a timely manner. Information can be collected and processed in just a few days. It also allows respondents more time to carefully consider response selection and to enter in text for open ended questions. If factual information is required then the respondent has enough time to consult their records. Advantages of Web-Based Surveys: Paper, postage, mail out, and data entry costs are almost completely eliminated; Time required for implementation can be reduced; Once electronic data collection system is developed, cost of surveying additional respondents is much lower; Reminders and follow-up on non-respondents are relatively easy; Data from Web-based surveys can be easily imported into data analysis programs; Page: 42/47

43 They are more inclusive, allowing a further reach then postal or phone surveys or direct interviews, potentially including a global audience. This can be useful in finding reasonable numbers of respondents with a rare condition; Once set up, they are cheap to carry out, making it easier to recruit large numbers of participants or to collect data repeatedly, on several occasions; The data are captured directly in electronic format, making analysis faster and cheaper. This again allows more data to be collected than with conventional mailed paper questionnaires; Associated material, such as data definitions or even the protocol for the study, can be linked to the data capture forms and vice versa; They allow interactive data capture with rapid checking of responses, at least at the form level; immediate validity checking of individual data items requires a Java applet; Web surveys allow the use of multimedia and enforced branching, and with Java applets they allow complex experiments with complete control over the scheduling of stimuli and responses without the need to mail each participant a floppy disk. However, there is a problem with measuring timing using simple HTML forms, as network response times are highly variable; Web surveys allow rapid updating of questionnaire content and question ordering according to user responses. Disadvantages of Web-Based Surveys The main disadvantage of web-based surveys, particularly for the public sector is the ongoing lack of internet access both within some geographical areas and within certain subgroups of the community; Inaccurate Demographic Data - unlike a study in which the researcher is interviewing a subject, online surveys depend on people to be honest about basic demographic information such as age, gender and race. Since people are not always honest, this can create inaccuracy in the data. Surveys that are sent to individuals who have been prescreened will not suffer from the same degree of inaccuracy; Technical Problems - occasionally, technical problems can affect the user experience, and subsequently the quality, of online surveys. Pages can time out and servers can become overloaded. Surveys can have technical glitches that are not apparent until Page: 43/47

44 significant errors begin to show up in the data. Individuals may be able to submit surveys twice, leading to errors in the data; The decision not to respond is likely to be made more quickly. The technical nature of online surveys is highly suitable for specialized or well defined populations that have access to an account and/or a computer. However, this alone is a disadvantage when measuring public awareness in ASO processes. Due to lack of internet access, availability of computers and lack of computer literacy, both within some geographical areas and within certain sub-groups of the community, use of web surveys for gathering data is not advisable when measuring public awareness in ASO process. On the other hand, use of web based tools is proposed for easier handling of data and interpretation of survey results. Therefore it is proposed to execute questionnaires and collect data using telephone based surveying and manual insertion of results in the web based tool for easier interpretation of results. There are several companies who provide online survey tools, with some of possible solutions presented hereunder: SurveyMonkey As with all of the survey applications SurveyMonkey lets users create surveys with a variety of question formats and get their responses, but SurveyMonkey also lets their users customize their surveys as well. Users are able to change thank you pages, upload a logo, or use one of the preset themes. Page numbering and question numbers are optional fields, and for users that want to save time a copy function allows for duplicating surveys. Page: 44/47

45 When collecting responses, there are three options: return by , create a popout, or set the responses to come to your . Most of the analysing of results is done under the My Surveys tab, which serves as a central point to design, collect, and analyse your surveys. Analysis shows response rates, the number of skipped questions vs. answered, and with a paid account, the ability to filter the responses (response time, or collector). Responses can also be shared without giving access to the account. Link: SurveyGizmo SurveyGizmo gives website owners a better way to conduct surveys of their visitors or clients. The application comes with many helpful features and users can choose from more than 20 question types which are ideal for research, tracking, marketing automation, and insight into what clients want. Surveys can be converted into reports so the user can review the information in a format that is readable and easy to understand. SurveyGizmo gives the user total control by Page: 45/47