VALIDITY. Alternate forms/internal consistency reliability = within a test - content validity = relationship between test content and external domains

Transcription

1 VALIDITY I. VALIDITY - test measures what it is supposed to measure - What a test measures and how well it measures it - What may be inferred or predicted from a test score II. TYPES OF VALIDITY A. Content - does the test cover good sample of the behavior/trait? - systematic examination of test content to determine whether it covers a good sample of the domain Alternate forms/internal consistency reliability = within a test - content validity = relationship between test content and external domains Achievement tests - requirements 1. Tests must adequately sample from the item universe 2. Tests must not over-represent aspects of the domain for which items are easy to construct 3. Tests must minimize the contribution of irrelevant factors to scores Content validity is built into a test by: 1. Surveying the universe of material 2. Systematically selecting items from this universe 3. Pilot-testing to detect problem items & contribution of irrelevant factors

2 Content vs. Face Validity - Face validity = what does the test appear to measure - Face validity can help with rapport B. Criterion - does test response correlate with another measure of the trait - the effectiveness of a test in predicting an individual s behavior - test scores are correlated with performance on a criterion 1. Concurrent - validating a test against a criterion collected at the same time 2. Predictive - validating a test against a criterion taken at some later point Issues 1. Is there a justification for developing this new test? - new test must be improvement over criterion measure 2. Criterion contamination - if scores on the test somehow come to influence the criterion, a falsely high correlation may appear between the 2 measures 3. Common criteria: Intelligence - Academic achievement Aptitude - Performance in specialized course of training Psychopathology - Psychiatric diagnosis 4. Tests also may be validated against existing psychological measures - new test must be better than criterion - exception = in construct validation

3 5. Reliability of the criterion creates a ceiling for criterion validity - quality of the criterion chosen is very important C. Construct - does test agree with the theory about the domain of interest? - the extent to which a test measures a theoretical construct or trait Construct validity is assessed by: 1. Postulation of the construct 2. Development of a test to measure the construct 3. Validation of the construct through demonstration of predicted relationships between test scores & other construct indicators Construct validity = broader and more abstract than other types of validity - subsumes the other types To demonstrated construct validity: 1. Developmental changes 2. Factor analysis - FA = reducing a complex data set to a smaller number of factors thought to underlie the other scores 3. Experimental data - test scores should show predicted changes in the underlying construct of interest

4 4. Multitrait-multimethod matrix - procedure to demonstrate the relationships among various assessment techniques and among various constructs Monotrait-monomethod - same trait measured by same method (Reliability coefficients) Monotrait-heteromethod = convergent validity coefficients - these correlations should be high - different types of evidence should converge on the construct Heterotrait-monomethod & heterotrait-heteromethod = discriminant validity - these correlations should be low - evidence from the same (or different) sources about different traits should be different -> to discriminate one trait from another Reliability = agreement between very similar methods of the same trait Validity = agreement between very dissimilar methods of that 1 trait

5 III. APPLICATIONS OF VALIDITY Validity data can lead to decision-making strategies - create cutoff scores on valid tests to make certain decisions Use of cutoff scores -> mistakes -> a cost-benefit ratio - which is a worse mistake for a particular type of decision? - decision about which mistake is more costly affects where cutoff is placed Validation of Rubin s (1970) Love Scale 1. Defined construct as attitude toward significant other - attitude -> thoughts, feelings & behaviors - attitude tied to specific person - romantic love among unmarried persons 2. Methodology a. Created items - loving & liking scales - judges sorted items (liking & loving) students took 70-item scale re romantic partner & friend - factor analysis -> liking & loving b. Established reliability - for revised, 13-item loving scale - internal consistency alpha =.84 for women,.86 for men - no test-retest -> good reliability

6 c. Established validity 1. Questionnaire study dating college couples - each completed re partner & friend Liking & loving moderately related.39 for women,.60 for men Both love partners more than friends Both liked their partners only slightly more than friends Love scores correlated with: how in love they felt how likely they thought they were to marry the partner - liking scores moderately related to in love likely to marry Loving not related to relationship length Loving not related to social desirability Loving symmetrical across couples - liking less symmetrical 2. Laboratory Experiment H 1 = in love -> eye gazing vs. couples less in love or unacquainted pairs strong love--together = 19 pairs weak love--together = 19 pairs strong love--apart = 21 pairs weak love--apart = 20 pairs