Predictive Validity of Aptitude Testing

Transcription

1 Lunova Group & Company A: Customer Sales Specialist (CSS) Analysis Customer Sales Specialist: 42 employees total (4 Team Leaders, 5 Trainers, 5 new hires) Testing Implemented: Criteria Basic Skills Test (CBST) & Customer Service Aptitude Profile (CSAP) Why Test? Research has shown that aptitude tests are the single most accurate predictor of job performance; they are twice as predictive as job interviews, three times as predictive as experience, and four times as predictive as education level, as shown in the chart below. 0.6 Predictive Validity of Aptitude Testing Predictive Validity ( r ) Cognitive Aptitude Tests Interviews Job Experience Education 1

2 Criteria Basic Skills Test (CBST) Description The Criteria Basic Skills Test (CBST) measures the basic math and verbal skills that are required to succeed in a wide variety of entry-level jobs. The CBST is a 20-minute, 40-item test that measures basic grammar, spelling, math, and language skills. It offers a quick way to assess the job readiness of candidates in a range of jobs, including clerical, administrative, and customer service positions. Score Reports Each individual is given a raw score and a percentile ranking. The raw score indicates how many questions (out of 40) the individual answered correctly, while the percentile ranking is a relative performance metric that indicates how the individual scored relative to others who have taken the test. For example, a percentile ranking of 45 means that an individual scored better than 45% of the group on which the test was normed. Validity Information The CBST has been shown to predict performance for a variety of jobs in which basic verbal and math skills are required, including customer service positions, data entry, sales clerks, and more. For example, in a sample of research associates at a market research firm, where research associates were required to conduct telephone-based surveys and perform data entry, there was an observed correlation of.26 between CBST performance and job performance. Job performance was measured by productivity ratings compiled by management and was compared to CBST scores. Individuals with failing CBST scores (raw scores less than 28) had an average productivity rating of 6.0, while those with passing scores had an average productivity rating of 9.8. In a second study, CBST scores were compared to job performance data for sales clerks at an apparel retailer with eight store locations. There was a.27 correlation between CBST scores and job performance. Standardization Sample Norms for the CBST were developed using a sample of 1,064 individuals. The sample was made up of working adults aged 18 and older. These individuals were being assessed for employee selection purposes, and represented a mixture of individuals from over 60 companies whose current positions included customer support representatives, factory workers, sales associates, and clerical personnel. The mean score for the standardization sample for the CBST was 30.9, the median score was 33, and the standard deviation is Customer Service Aptitude Profile (CSAP) 2

3 What is a Customer Service test? The Customer Service Aptitude Profile (CSAP) is an adaptation of the Sales Achievement Predictor and measures personality traits that are critical to success in customer service and customer service-related positions. The report describes characteristics related to customer service potential and performance, such as diplomacy, cooperativeness, patience, and assertiveness. The customer service test contains 140 items and is written at a sixth-grade reading level. Validity scales identify applicants who exaggerate strengths or minimize weaknesses and automatically adjusts the scores of those applicants. Detailed Score Reports The CSAP presents an overall recommendation about an individual's suitability for customer service work, as well as a listing of his or her relative strengths and weaknesses in eight skills and behaviors associated with good customer service performance. Because many customer service roles include a sales component, specific scores that focus on a person's disposition towards salesrelated activities are also provided. Validity Information Construct Validity: CSAP scale scores have been compared with a number of external measures of similar individual characteristics, including the Sixteen Personality Factor (16PF) and the Edwards Personal Preference Schedule (EPPS) scales; these comparisons provide very strong support for the construct validity of the CSAP scales. Predictive Validity: The CSAP overall score is based on a combination of personality traits that have been shown to be critical for success in customer service and customer service-related jobs. These traits include cooperativeness, personal diplomacy, and patience. CSAP scores have been shown to correlate with job performance for customer service representatives in a wide variety of industries, and can also be used to screen candidates for related positions such as bank tellers, front-of-house workers in the restaurant industry, receptionists, and more. Reliability: The estimated test-retest reliabilities for CSAP scores range from.67 to.90 and are well within acceptable psychometric limits. Standardization Sample Norms for the CSAP were developed using a sample of 1,375 individuals aged 15 and older, of which 48% were male and 52% were female. These individuals were being assessed for job selection, placement, promotion, or career appraisal. Approximately 30% were students and the balance were adults ranging in age from their early 20s to mid-60s. Their current work situations included executive positions in technical and service-oriented areas, managerial positions, and sales positions. About 10% of the sample were Black and 2% were Hispanic. In establishing standardized scores for the final version of the CSAP, normalized T-scores were first developed on the large, stable standardization sample for CSAP scores. The normative information obtained from the standardization group was extended to a separate reference sample of 745 managers, people in sales-related occupations, and middle- and upper-level professionals. 3

4 Criteria Basic Skills Test Results Raw Score Percentile Average % Average Top % Average Top 5 (non management) % Average Bottom 5 (not new) % Average Management % Average Trainer % Average (non management) % Percentile CBST Percentile

5 CBST Score Distribution Excellent (38 Plus) Good (31-37) Fair (28-31) Low (below 28) What can we learn from this data? None of the CSS who scored in the Excellent range for aptitude rank in the top 38% of performers. The average for CSS District Manager aptitude is approximately 15% higher than the average for non managers. Those that scored Low on the CBST all rank in the bottom 40% of performers. Our ideal range for a successful CSS falls in the Fair to Good range of on the CBST scale. With a limited number of District Managers, we are not able to target the exact CBST range that is ideal, but all four leaders fall in the Good range and are ranked in the top 20% of performers. There is a high likelihood that is the ideal target range for a District Leader. 5

6 Customer Service Aptitude Profile (CSAP) Results **Note All 42 candidates were recommended or highly recommended for CS **These averages do not include management or new employees. Sales Disposition Initiative - Cold Calling Sales Closing Achievement Average All Average Top Average Bottom Motivation Competitive -ness Goal Orientation Planning Initiative - General Average All Average Top Average Bottom Team Player Managerial Assertiveness Personal Diplomacy Extroversion Average All Average Top Average Bottom Cooperativeness Relaxed Style Patience Self-Confidence Average All Average Top Average Bottom

7 Self-Confidence Patience Relaxed Style Cooperativeness Extroversion Personal Diplomacy Assertiveness Managerial Team Player Initiative - General Average Bottom 5 Average Top 5 Average All Planning Goal Orientation Competitiveness Motivation Achievement Sales Closing Initiative - Cold Calling Sales Disposition

8 Personality Traits - All CSS Team Personality Traits What can we learn from this data? A sales profile is not necessary successful in the CSS role. The team itself has a very low average in regards to Sales Closing and actually top performers score 15% lower than average in this category. Not surprisingly, Personal Diplomacy is clearly the most important characteristic of a successful CSS. The top five performers scored 12% higher in Personal Diplomacy than the bottom five, and 10% higher than the average CSS. The ability to restrain assertiveness is also a key factor in determining who will be a successful CSS. The top 5 CSS performers scored over 10% lower in the Assertiveness category than the average and bottom five CSS. Being a team player comes up as one of the lower scores in the CSS team. It is important to note we are not looking for someone that scores highly on the scale, but the top five ranked CSS scored almost 20% higher in this category than the bottom five. Cooperativeness is also a major factor in a successful CSS. The top five scored approximately 10% higher than average in this category and the personality trait is the third highest factor among all of the CSS team. Motivation and a focus on getting things done is also an important trait for the CSS team. It ranks second highest as a trait and the top five scored several points higher than average in this category. 8