MRS Diploma in Market & Social Research Practice

Transcription

1 Exam Paper No: MRS Diploma in Market & Social Research Practice UNIT 1 The Principles of Market & Social Research Examination Paper, Answer Guide & Sample Answer Wednesday 4 th June am 12.30pm Instructions for Candidates Time allowed 3 hours. Candidates must answer all questions within this exam paper. Question 1 is worth 50% of the final mark and Questions 2 and 3 are each worth 25%. Where questions are subdivided into 2 or 3 tasks, the weighting for each task within the question will be clearly indicated. All answers must be written in your Examination Answer booklet. Important Note: The requirement is for candidates to complete questions 1, 2 & 3 and all part questions within these questions, failure to do this by not completing the requirement may result in the paper being marked as non-compliant. This Exam Paper must not be removed from the exam room. At the end of the exam please hand this paper back to the Invigilator. The research problems contained in this material are fictional, any similarity to any real-life organization, company or business is entirely unintended.

2 With members in more than 60 countries, MRS is the world s leading authority on research and business intelligence. MRS Diploma in Market & Social Research Practice Unit 1 4 th June 2013 Examination Paper, Answer Guide & Sample Answer The Market Research Society 2013 MRS The Old Trading House 15 Northburgh Street London EC1V 0JR Telephone: +44 (0) Fax: +44 (0) profdevelopment@mrs.org.uk Website: Company Limited by guarantee. Registered in England No Registered office as above. Page 2 of 16

3 Read the following case study and answer ALL questions. Drivefast The Market Research Society 2013 Drivefast is a company that provides road freight services to 190 manufacturing and retailing companies in the UK. These are split into the following broad categories: 90 manufacturers of grocery products 10 manufacturers of car parts 40 manufacturers of industrial products 20 manufacturers of clothing products 10 manufacturers of building products 5 grocery retail chains 15 high street retailers Around 20% of these companies use Drivefast for all of their consignments, 50% use Drivefast for a proportion of their assignments on an ongoing contractual basis and the remainder use Drivefast interchangeably with other road freight suppliers on one-off consignments. Drivefast revenues have been decreasing over recent years, so the company wishes to improve the customer experience through the various touchpoints that it has with its existing client companies. These touchpoints relate to the booking of a consignment pick-up, the collection of consignments, the delivery of consignments, invoicing and the interactions with the customer service department when things go wrong. The company wishes to better understand the experiences that the client companies have had at each of these touchpoints in terms of the areas where they are satisfied and the difficulties they have encountered. The company is also keen to explore client suggestions for improvement / changes in the procedures and operations of Drivefast. The company hopes that such information will assist Drivefast in improving their customer experience management resulting in higher market share and greater revenues. The company is willing to spend up to 60,000 on the research project. Page 3 of 16

4 Question 1 You are a Research Executive in a market research agency and have been asked to make reasonable assumptions based on the brief and produce a proposal to address the research needs of the company.¹ 1 For the purpose of this question, your proposal should exclude the sections relating to personal CVs, related experience and references and contract details. (Total 50 marks) Answer Guide Candidates are expected to produce a proposal in a standard format. The exact content will be determined by the assumptions made by each candidate. These should include: Background (8 marks) Describe the company, its activities, the marketing decisions Drivefast required to make and the rationale for doing the research. This should be done in an appropriate style and should be written in clear, non-ambiguous language. Objectives (16 marks) Objectives should be more precisely defined than they were in the original brief. They should clearly demonstrate an understanding of the information needs in relation to the marketing decisions that are likely to be undertaken by Drivefast. Approach and Method (20 marks) A whole range of qualitative and quantitative approaches could be suggested for choices made, candidates should demonstrate that they have taken account of: the objectives the target market the size of the budget This section should clearly set out how the information is to be gathered and the sampling approach. Better answers will set out justification for these choices. Reporting and Presentation Procedures/Timing Fees (6 marks) reporting and presentation procedures timing fees This last section will not be assessed for accuracy of calculations/suitability of figures/timescales but candidates will be rewarded for including them and setting them out in an appropriate format. Page 4 of 16

5 Sample Answer Dear Research Manager at Drivefast, Here is our proposal for the customer experience management project. Please don t hesitate to contact us if you have any questions. Background score: 6/8 Drivefast is a road freight service company that operates in the UK and has 190 manufacturing and retailing clients: 90 manufacturers of grocery products 10 manufacturers of can parts 40 manufacturers of industrial products 20 manufacturers of clothing products 10 manufacturers of building products 5 grocery retail chains 15 high street retailers 80% of these clients use Drivefast for a proportion of their assignments or for one-off consignments. Drivefast wish to increase its revenues and market share through improving its customer experience management of all the touch points its clients experience (bookings, collections, delivering, invoicing, customer service), to encourage these client companies to increase the proportion of assignments they award to Drivefast. In particular, the Customer Experience and Operations teams sponsoring this research want to be able to target the highworth companies, whilst Account Mangers want tactical information to improve their relationships with their client contacts. Objectives score 9/16 There are a number of key objectives that this research has to meet: To understand where Drivefast is satisfying and failing to satisfy its clients along the various touch points: - Booking of a consignment pick-up - Collection of consignments - Delivery of consignments - Invoicing - Interactions with the customer service department To understand the relative importance of the touch points in driving overall satisfaction and thus determining where Drivefast should concentrate its efforts to make the greatest impact - In particular, to focus on Drivefast s high-worth target group To understand the correlation between satisfaction and the intent to commission more assignments To gather feedback from client contacts which can be given to their Account Managers To gather suggestions for improvements in the procedures and operations of Drivefast and to understand how popular these changes might be. Page 5 of 16

6 Approach and Method score 16/20 In order to meet the needs of this research we suggest a two-stage approach: 1. An exploratory qual phase which includes idea generation. 2. A descriptive quantitative phase which quantifies the findings from stage 1. Stage 1: A series of 1.5 hour long focus groups which will allow us to explore the various touch points of the customer experience with the client contacts. We suggest conducting the following: 2x groups where the client contacts use Drivefast for all their consignments (roughly 38 companies). We will conduct one group each in Manchester and London as from your contact details list this would seem to be most convenient for the client companies. Two groups is really the maximum we can expect to recruit (10 recruited for 8 to turn up) from such a small sample 4x groups where the clients use Drivefast for a proportion of their assignments on an ongoing contractual basis. 2 groups per location. Again the number of groups is limited by the number of clients (roughly 80) 3x groups where the clients use Drivefast interchangeably with other suppliers on a one off base (from roughly 57 companies). 1 group in Manchester, 2 in London. All of these groups will be made of mixes of category types, since focus groups are an excellent way of hearing reviews as the views from the small number of grocery retail chains may not be able to be covered in the groups as they may refuse or not be available to take part. However, we will use judgement sampling to target those respondents who would be most valuable to hear from the high-worth targets which include these retail chains. Those who are not heard from in the stage will have a chance to be heard from in the quantitative stage. To facilitate recruitment, we ask Drivefast to contact its client contacts by their preferred means (telephone/ ) letting them know that this research is about to take place, encouraging them to take part to help Drivefast improve its CEM, and informing them that we will be contacting them to set up the research. We will then recruit the client contacts via their preferred method (telephone/ ). The focus groups will also be helpful for generating ideas for improvements, respondents can bounce ideas off each other and project techniques can help facilitate. The results from the focus groups will be made into transcripts. These will be coded up and content analysis software will be used to draw out the common phrases which we can test in the quantitative phase, like any drivers of satisfaction or dissatisfaction. Throughout the quantitative phase analysis of the qualitative results will continue using the cut and paste methodology to explore key themes. Page 6 of 16

7 Stage 2: A quantitative survey that will be sent to all client contacts, as there are only 190 contacts we will try a census because it will allow us to get a very thorough and representative view. Because we are trying to contact all the clients, we need to do everything we can to encourage response rates. There are a number of ways we can do that: Drivefast can encourage to warm up respondents as mentioned in the qual stage We can offer the respondents two options of taking the survey either via CATI so that they can set an appointment at a time convenient to them or an online version of the survey so they can complete it whenever they want. We can offer incentives We offer respondents the option of having their responses passed back to their Account Managers. Some may want their responses and attributed remarks passed back so that improvements can be made. Others who refuse permission can feel confident that their confidentiality will be upheld. (Similarly, for the stage 1 qual, respondents will be warned when recruited that there might be observers at the focus groups they know, so they have the right to refuse! The survey will be short so it doesn t take up too much time We can use call backs and reminder s to encourage response The survey will cover the factors of satisfaction and dissatisfaction as revealed in the qualitative phase. We can use factor analysis to understand the interrelationships between the results and multiple regression analysis to see which of these factors drive overall satisfaction and intent to commission. We can also use a conjoint analysis to determine which of the suggestions for improvements are most important, especially for high-worth clients. Reporting and presentation procedures score 5/6 We will conduct an interim workshop with key stakeholders of Drivefast after the qualitative stage to discuss the findings and formulate questionnaire wording. This is important because it means that in the questionnaire we are only testing ideas for improvement that are liable for Drivefast to put into place. At the end of the research we will provide Drivefast with an ed PowerPoint report and we will come to the head office for a face-to-face presentation. Timings After receiving confirmation that we have won the project we suggest these timings: Drivefast warm up client contacts and we write the focus group discussion guide = 1 week Sign-off of discussion guide and recruitment = 1 week Qualitative fieldwork = 1 week Initial analysis, interim meeting with Drivefast and writing the questionnaire = 3 weeks Quantitative fieldwork (including reminders and call backs) = 3 weeks Analysis and report writing = 2 weeks We will decide a final presentation data in conjunction with Drivefast Total length of project = 11 weeks Page 7 of 16

8 Cost We have broken down the costs as so: Stage 1: 9x focus groups (including recruitment and incentive, moderation time and travel costs, discussion guide writing and initial analysis, printing costs for projective materials, viewing facility hire) = 30,000 Interim workshop (including travel to and from the workshop) = 3,000 Stage 2: 190x quantitative 15 minute interview (including incentives, scheduling of appointments, call backs and reminder s, questionnaire writing, CATI cpi, programming fees, data processing) = 17,000 Analysis and reporting (including statistics) 7,000 Final presentation (including travel) 2,000 Total cost = 59,000 Kind regards, The Researcher Overall score: 36/50 Page 8 of 16

9 Question 2 a) Design a discussion guide for use in a series of group discussions to address the research objectives of the proposal set out in Question 1. (15 marks) b) Provide a detailed description of two projective techniques that could also be used within the group discussions, alongside the discussion guide, to help address the research objectives. (10 marks) Answer Guide (Total 25 marks) The discussion guide and projective techniques should be assessed on: appropriate phases and ordering of the discussion guide: Introduction o Objectives o Personal Introductions o Agenda Discussive o Topic Areas o Stimulus Material Summarising o Summarising Discussion o Closing o Administration (5 marks) appropriateness of topics to the research objectives. (10 marks) detailed description, creativity and appropriateness of projective techniques to research objectives. Examples may include brand mapping, brand personalities, projective questioning, cartoon completion, mood boards, sentence completion, story completion, word association, role play, photo sorts etc. (10 marks) Page 9 of 16

10 Sample Answer Question 2a score 12/15 Objectives for the moderator to cover off in the focus group: What are the strengths and weaknesses of Drivefasts customer experiment offer along all the touch points where the client companies might come into contact with Drivefast Understand which are most important in driving overall satisfaction and intent to continue using Drivefast or to use it more Gather ideas for improvements Understand any differences between client companies in different categories (manufacture verses retailers, etc) It is important that you hear from everyone in the group, especially any one from a highworth target company. Discussion guide: Introduction (10 minutes) Introduce yourself Welcome them to the group Remind everyone that this group is being conducting according to the MRS Code of Conduct, including their rights to confidentiality Check for consent to record the group. Also remind respondents that they are being observed by someone who may know them. Explain what this means vis a vis confidentiality and inform them of the identity of the interviewers so that they can decide whether the stay in the group or leave. Explain the format general subject matter and purpose of the group, as well as the length Get respondents to introduce themselves including their names, category of the company and their relationship with Drivefast. Usage of road freight services (20 minutes) Explore how respondents use Drivefast and its competitors Who do you use for road freight services? Probes: - Why? - How long have you used them? - How often? Introduce word association exercise about the road freight companies used. What would you say are the main differences between the road freight companies you use? Probe on differences between Drivefast and others. Strengths and weaknesses of Drivefasts CEM (30 minutes) We re now going to explore Drivefast in particular. How do you have contact with Drivefast? Probe for booking collection, delivery, invoicing, and customer service. What works well and what doesn t work as well with each of those stages? Probe for each stage Which of those stages are most important for Drivefast to get right? Probe - What drives overall satisfaction Page 10 of 16

11 - What makes them most likely to want to use Drivefast again The Market Research Society 2013 The ideal road freight company (25 minutes) What could Drivefast do to improve their weaknesses? Which of these suggested improvements would be most valuable Introduce role playing exercise to demonstrate ideal customer service What would your ideal road freight company look and act like? Summary and thank you (5 minutes) Please tell me the 3 biggest weaknesses for Drivefast Please tell me the 3 best improvements Thank respondents for attending and make sure they sign the incentive form. Page 11 of 16

12 Question 2b score 8/10 2 projective techniques: 1. Word association: This helps respondents to warm up in a focus group by being a fun exercise that allows them to speak up in a low-pressure environment and interact with other respondents. It works by asking respondents to think of one word they associate with, in this case, each road freight service they use. The moderator would introduce a company s name, like Drivefast, and ask respondents to go round in a circle giving one world in response. The exercise can then be repeated using a different order of respondents with another company. The moderator can also start probing respondents on why they used each word which helps bring to light opinions respondents might not necessarily expressed without the exercise. It also makes sure that the moderator hears from everyone 2. Role-playing exercise: This is a good exercise when respondents have been warmed up and are happy to talk and perform in front of others. It can be used by splitting respondents into pairs and asking them to act out a scenario. The can practise at first for 5 minutes and then present back to the group, where afterwards they can explain their actions and the rest of the group can comment. It would be especially useful in this focus group because the moderator could give each pair a common problem scenario that companies have with their road freight services and then the pairs can act out the ideal solution. Overall score: 20/25 Page 12 of 16

13 Question 3 The senior executives of Drivefast wish to be prepared for assessing the research proposals that they are likely to receive, and are concerned about their lack of knowledge of respondent sampling and issues of anonymity. They have asked Drivefast s Insight/Research Manager to write a paper clarifying: a) What is meant by sampling and the considerations researchers use when determining the size of a sample taking part in a research project. (15 marks) b) the extent to which Drivefast can get information about actual respondents and their organisations, in addition to feedback on the specific issues that they raise so that the information can be matched against specific clients and their consignments/deliveries. (10 marks) Answer Guide (Total 25 marks) For part a), Candidates should define sampling and include factors that impact on sample size such as: budget sample sizes used in similar previous studies the homogeneity of the population of interest the likely response rates the incidence rate of the characteristic being investigated the number of subgroups of data that are to be analysed (15 Marks) For part b), Candidates should identify the ethical/mrs Code of Conduct issues relating to a researcher revealing a respondent or passing on attributable comments, such as: The anonymity of respondents must be preserved unless they have given their informed consent for their details to be revealed or for attributable comments to be passed on. In particular with this case, care should be taken if sample sizes are very small (such as in business and employee research) that they do not inadvertently identify organisations or departments and therefore individuals. If respondents have given consent for their data to be passed on in a form which allows them to be personally identified, the researcher must: o Demonstrate that they have taken all reasonable steps to ensure that the data will only be used for the purpose for which the data was collected; and o Fully inform respondents as to what will be revealed, to whom and for what purpose. If respondents request individual complaints or unresolved issues to be passed back to Drivefast, the researchers must comply with that request. However, the comments/issues to be passed back to Drivefast must be agreed with Respondents and must not be linked back to any research data, or used for any other purpose, without the explicit consent of the Respondent. (10 marks) Page 13 of 16

14 Sample Answer Question 3a score 10/15 Sampling is how researchers select who they are going to interview for a particular project. There are multiple stages: 1. Deciding the population of interest. This is usually the client s target market. It is who the research should be focused on e.g. Smartphone owners in the UK for a Smartphone case manufacturer 2. Decide whether to sample census. Census is when you want to interview all of the population of interest but is usually meant to be representative of the whole. 3. Chose a sampling frame. A sampling frame is a list of potential respondents and their contact details. It is supposed to represent all of the population of interest and can be a list of people (i.e. Client database, telephone directory) or of organisations, or of locations (e.g. Postal Address File) 4. Choose the sampling method: How you select the sample has a great impact on determining sample size. There are two main types of sampling methods. The first is probability sampling, where every member of the population of interest has a known chance of being selected. The second is nonprobability, where the chance of selecting each member of the population of interest is unknown. Sample sizes for probability sampling are usually larger than for nonprobability sampling, because they are more concerned with being representative of the population of interest and reducing sample error (the chance differences with the survey mean and other results caused by biases introduced because the sample is only a selection from the whole population of interest). 5. Decide sample size: For probability sampling, equations are available to determine sample size, which involve using statistics but also a researcher s subjective input. For example, simple random sampling uses this equation to determine sample size (n): N = Z 2 o- 2 E 2 Where Z = the confidence level required. Usually 95% confidence level (which means that the results of the survey will have a 95% chance of reflecting the true population mean + sample error) is used in market research, so Z = 1.96, though Z = 2.58 can be used for 99.7% confidence. E = the level of precision i.e. how close does the sample mean have to be to the true mean. If the client needs a very precise figure then this sample error will be small o- = the standard deviation, which indicates how for the average unit is away from the population mean. Because the research is usually being conducted to find this out the researcher has to choose a reasonable stand-in based on: Previous similar research which will might have a standard deviation calculated or give an idea of how much variances there is in the sample Secondary research might suggest a number A pilot testing of the survey amongst the same population of interest will give a number The researcher s own experience. Page 14 of 16

15 Of course time and cost increase in direct proportion to sample increase in direct proportion to sample increases for both probability and non-probability samplings, so these might impose limits of what sample size is practical (sample error can be reduced by adding sample in probability sampling but only at the rate of the square root of the proportion of increase of the sample). Other issues true for both probability and non-probability is that sample size needs to be big enough that subject can be examined. For non-probability sampling like convenience, judgement and quota which tend to be more subjective, sample size can be determined by a more subjective decision from the researcher on how many respondents are needed to represent the key focus of the research Question 3b score 9/10 The MRS Code of Conduct has very strict rules on anonymity and confidentiality. The respondent has a right to anonymity, that is, to not having any personal data and details or any attributable comments being passed back to the client or to a third party. However these details and comments can be passed along if the respondent has given informed consent. This means that the respondent has been told what information has been given to whom and for what purpose. The research agency also has a responsibility to demonstrate that they have taken all reasonable steps to make sure that the data and comments are not used for any other purpose than that which was stated to the respondent so in the case of the Drivefast research we would have to ask the permission of the respondents to pass on their attributable comments back to their account managers for the purpose of improving the service offered to the specific clients. But the comments should only be used for that purpose, and not for, for example, marketing materials for Drivefast. In some cases, especially customer service research like this one, respondents ask for their feedback to be returned to the client. This is usually the case when a respondent has a complaint to make. The researcher has an obligation to pass the comment back to the client, having agreed with the respondent exactly what the comment says and what respondent details will be passed along with it (usually name and contact details). In these cases Drivefast will be given all the comments that respondents specifically request be passed on to it. In all other cases and without respondent permission, the client is only allowed to see anonymised data. Overall score: 19/25 Page 15 of 16

16 MRS The Old Trading House 15 Northburgh Street London EC1V 0JR Telephone: +44 (0) Fax: +44 (0) Website: Company limited by guarantee. Registered in England No Registered office as above. Page 16 of 16