PSYCH 412: Psychological Testing and Measurement Fall 2014 Tuesdays & Thursdays 2:50-4:05pm VMMC 219

Transcription

1 PSYCH 412: Psychological Testing and Measurement Fall 2014 Tuesdays & Thursdays 2:50-4:05pm VMMC 219 Course Instructor Tahira Probst, Ph.D. Industrial/Organizational Psychology Office: VCLS 208K Telephone: (360) Office Hours: Tues/Thurs 9-10am and by appointment Course Website The Psych 412 website is available to all enrolled students via Angel at This syllabus, the course assignments and drop boxes, grades, and any course announcements will be posted there. So, be sure to check it frequently and/or set up notifications to ensure you do not miss any important information. Required Instructional Materials Kline, T. (2005). Psychological Testing: A Practical Approach to Design and Evaluation. Sage: Thousand Oaks, CA. May be purchased through the Bookie or an online source. Catalog Course Description PSYCH 412 Psychological Testing and Measurement 3 Course Prerequisite: PSYCH 311. Assessment of behavioral variables in humans; individual differences. Recommended preparation: PSYCH 105. A Better Description In this course, we will review the principles of psychological assessment, including test development, psychometric principles, and applications in industrial/organizational, educational, and clinical settings. A major portion of the class will be devoted towards the development and validation of a test or measure that you will design yourselves. This course will be excellent preparation if you are intending to go to graduate school in psychology, education, or another related field. This class is also extremely useful if you plan to go into HR, where you will frequently be involved in hiring new employees, constructing employee and/or customer surveys, and developing measures to assess employee performance. Two Caveats (1) This is an advanced upper-level undergraduate course and you need to have a firm foundation in statistics to succeed in this class. If you have not taken Statistics (Psych 311) yet, you need to drop this class and re-enroll after you have completed the Psych 311 pre-requisite. (2) Students often think this class is the same as Psych 312 Research Methods it s not! You will not be conducting any experiments. However, you will be developing a measure that could be used in a future study or experiment. It is a very different process. Specific Course Learning Outcomes and Assessment Because Psych 412 focuses on the scientifically-valid development and ethical usage of psychological testing, it provides a unique opportunity to become more proficient in each of the seven WSU Learning Goals: 1) Critical and Creative Thinking, 2) Quantitative Reasoning, 3) Scientific Literacy, 4) Information Literacy, 5) Communication, 6) Diversity, and 7) Depth, Breadth, and Integration of Learning. You will achieve these course aims primarily by developing an assessment or test of your own from start to finish. Thus, you will be responsible for identifying a construct you want to measure, writing items to measure the construct, collecting data to assess the reliability and validity of the measure, and revising your scale based on analyses that identify good versus poorly written items.

2 WSU Learning Outcomes Quantitative Reasoning Course Learning Outcomes and Methods of Assessment Course topics that address these learning outcomes are: At the end of the course, (see calendar for specific topics, This outcome will be students should be able to: dates, and associated assignments) evaluated primarily by: Grasp basic psychometric Statistics Refresher Quizzes properties (standardization, Validity Paper reliability, validity) involved Reliability Poster in psychological assessment Test Development Project Diversity Depth, Breadth, and Integration of Learning Scientific Literacy Information Literacy Critical and Creative Thinking Communication Conduct statistical analyses and properly interpret the results Be aware of the ethical, legal, and social issues as they relate to test theory and application Understand the issues related to practical applications, including test administration and evaluation in a variety of settings Develop an assessment or test of your own from start to finish. This will involve: a literature search identification of construct to measure item writing collection and analysis of data to assess the reliability and validity of the measure, and interpretation of results including suggestions for revising your scale based on analyses that identify good versus poorly written items Present the results of your test development project both in writing and orally. Ethics and Professional Issues Career Assessment Review of Commonly Used Tests in Psychological, Educational, and Organizational Settings Test Development Project Test Development Project Construct Cleanliness / Literature Search Item Writing Designing and Scoring Responses to Items Data Collection Item Analysis Scale Revision Test Development Project with opportunities for feedback on rough drafts throughout semester Quiz Quiz Paper and poster Paper Poster Quizzes Submission of rough drafts APA-style Paper Poster presentation

3 Course Requirements In general, students get out of a course what they put into it, meaning the more YOU are able to dedicate to attending class consistently and completing course requirements, the more you will gain. If you feel your participation in this class may be compromised by a heavy schedule or life circumstances, consider taking it at a different time. The reading load and course project will be rewarding, but somewhat time-consuming. 1. Assigned readings and class participation. Please read the text and assigned readings for a particular lecture before coming to class. Because this is a senior-level course, I expect students to take an active role in discussing course readings and lecture material. Also, there will be many in-class demonstrations which will be much more successful to the extent that you are familiar with the material. I will warn you: the textbook is no Harry Potter. It can be pretty dense stuff. That s why it really helps to read it before class, so you will better understand the lectures and can ask questions about the reading that you didn t understand. Also, this is a very hands-on course you will be working with your classmates to apply the text and lecture concepts to your scale development project. But, you can t help each other if you haven t read the material. 2. Quizzes. To assess your understanding of the readings and other course information, there will be five, 20-point quizzes throughout the semester. Quiz dates are on the attached schedule. Quiz questions are from class lectures, readings, and in-class demonstrations. There will be no late or make-up quizzes during the semester. If you are ill or must miss a quiz for any reason, there will be a make-up quiz offered at the end of the semester that will be comprehensive (i.e., will cover material from the entire semester). If you miss a quiz, you must take the make-up exam. If you are present for all 5 quizzes and are satisfied with your grade, you do not need to take the make-up quiz. However, if you wish to attempt to improve your grade, you may take the make-up quiz and substitute that quiz for a lower quiz grade. If you perform worse on the makeup than on your lowest quiz, I will just pretend I never saw the makeup exam. 3. Test development project. For this project, you will develop, administer, and critically evaluate your own psychological test. At the end of the semester, you will hand in an APA-style paper describing the development and validation of the measure and present a poster of your findings to your classmates and psychology faculty. Your individual project is worth 120 points (100 for the paper and 20 for the poster). Your psychological measure can assess any construct of your choice. A sample of possibilities includes: Test Anxiety Emotional Intelligence Jealousy Stress Introversion/extraversion Job Burnout Leadership Conservatism/Liberalism Depression IQ Morality Cultural Values Many other possibilities exist the most important thing is to find a topic that you are really interested in. After you develop a pilot version of your test, you will collect data from your classmates during our data collection day mid-semester. Next, you will enter the data into SPSS, and conduct the appropriate statistical analyses to explore the psychometric properties of your test (e.g., descriptive statistics, reliability, validity, etc.). Based on your analyses, you will write an APA-style manuscript in which you describe the psychometric qualities of the instrument. The paper MUST be written in APA-style format. If you are not familiar with APA-style, you might check out this web site for help: or take a look at the Publication Manual of the American Psychological Association (6th Ed.) in the library. Finally, during the last week of classes you will present a poster of your findings. This presentation is worth an additional 20 points; faculty and graduate students from the Psychology department will be invited.

4 Below is the required structure for the report along with specific criteria for elements that should be contained in each section. There are no minimum or maximum page limits for the paper, although typically these kinds of papers usually end up being pages of text, plus References, Tables, Figures, and Appendices. The timetable for the completion of the various parts of your project is laid out in the Course Calendar. I. Introduction Introduce the topic Define the construct to be measured Discuss why this topic is relevant / interesting Describe other measures that exist to assess this construct Explain how you propose to validate your measure (i.e., shows it measures what you think it measures); this will include one or more hypotheses (e.g., if my new test measures IQ, then scores on it should be positively correlated with other validated measures of IQ). II. Method Test Development (Describe the development of the test. Make sure the reader understands how the items were generated, what the item format is, etc.) Other measures included in the study (for purposes of validity assessment) Describe Participants (sample size, recruitment, age, sex, etc.) Procedure (e.g., data collection, description of proposed analyses) III. Results Item Analyses (Examine frequency distribution of individual items, item means, standard deviation, item-total correlations) Reliability Analyses (Report reliability estimates, such as Cronbach s alpha). Validity Analyses (Examine correlations between your measure and the validation measures. Examine whether the correlation patterns follow theoretical expectations.) Proposed Scale Revisions (based on above analyses) IV. Discussion Summarize your main findings regarding the reliability and validity of your scale. Suggest additional analyses and research that would help you to better understand and/or improve the reliability and validity of the test Describe the practical uses of your test (who are the potential users of this test) V. References List all books, articles, and online sites you have cited in the text. VI. Tables and Figures VII. Appendix Append copies of your scale and any other scales used for validation.

5 Grades The grading policy is as follows: Five 20 points each: 100 points Final Paper: 100 points Poster Presentation: 20 points TOTAL POSSIBLE: 220 points Grades will be determined as follows: Grade Percent Total Number of Points A B C D F <60 <132 There will be no rounding up and no pluses or minuses. Midterm grades will be submitted to the Registrar during the eighth week of the semester, although up-to-date interim grades can always be found on Angel. Attendance Policy There is no mandatory attendance. However, your acquired knowledge, ability to complete assignments, and likely your final grade will all suffer if you do not regularly attend class. Reasonable Accommodation Accommodations may be available if you need them in order to fully participate in this class because of a disability. Accommodations may take some time to implement so it is critical that you contact Disability Services as soon as possible. All accommodations must be approved through Disability Services, located in the Student Resource Center on the Lower Level of Student Services Center, (360) Academic Integrity Academic integrity is the cornerstone of the university and will be strongly enforced in this course. The penalty for violating the honor code is severe. The first offense will result in a grade of zero on the assignment/exam in question. A second offense will result in a failing grade for the course. All offenses will be reported to the Office of Student Affairs. If a student is unclear about whether a particular situation may constitute an honor code violation, the student should meet with the instructor to discuss the situation. For additional information about WSU s Academic Integrity policy/procedures, please contact (360) Emergency Notification System WSU has made an emergency notification system available for faculty, students, and staff. Please register at zzusis with emergency contact information (cell, , text, etc.). You may have been prompted to complete emergency contact information when registering for classes. In the event of a building evacuation, a map at each classroom entrance shows the evacuation point for each building. Please refer to it. Finally, in case of class cancellation campus-wide, please check local media, the WSU Vancouver web page and/or Individual class cancellations may be made at the discretion of the instructor. Each individual is expected to make the best decision for their personal circumstances, taking safety into account. Laptop and Cell Phone Usage Laptops may only be used in class for typing lecture notes or conducting analyses in SPSS. They may NOT be used for IM, , surfing the web, updating Facebook, or any other non-course related activity, as this can be very distracting to other students. Anyone who violates the laptop policy will no longer be allowed to use their laptop in class. Cell phones must be turned OFF before class starts. Anyone whose phone rings during class will have 1 pt. docked from their next quiz for each instance.

6 Project Grading Criteria PAPER (100 points) Element Format APA style -- cover page, appropriate subheadings, etc. Total points Spelling/Grammar/Quality of Writing (e.g., flow and structure) 10 points Note: these are not freebie points! If your writing is poor, you can receive a full grade lower even if the project itself is perfectly conducted. Body of Report: (grading based on coverage of elements described in Project Description) Introduction Methods Results Discussion References 10 points Tables and Figures 5 points In addition, throughout the semester, I am requesting that you submit (via Angel) rough drafts of each section of your paper (i.e., Introduction, Methods, Results, and Discussion). This way, I can provide you with feedback so the final paper will hopefully be improved. Handing in the rough draft of each section on time is worth 3 of the 15 points for each respective section. Grading Based on Poster Coverage of: Purpose / Hypotheses Method Results Discussion Figures/Tables General Presentation (i.e., neatness, clarity) POSTER PRESENTATION (20 points) I will bring to class a few examples of conference posters, so you can see what a good poster should look like.

7 Course Calendar Date Topic Reading Assignments 8/26 Syllabus Review / Introductions 8/28 What is this class about anyway? Construct Cleanliness 9/2 Stats Refresher I Scales of Measurement; Central Tendency 9/4 Stats Refresher II Distributions; Statistical Significance; Norms Ch. 1 pp. 1-4, Ch. 1 pp. 4-5 Ch. 1 pp. 5-7; /9 Stats Refresher III Correlation; Regression; R 2 Ch. 1 pp /11 QUIZ 1 9/16 Rules for Writing Items Ch. 2 (skip pp ) 9/18 Designing and Scoring Responses to Items Ch. 3 (skip pp ) Choose Project Construct ( instructor with ideas) 9/23 Scale Development Day Bring in Your Draft Scale Items 9/25 Collecting Data Ch. 4 (skim) 9/30 QUIZ 2 Rough Draft Introduction Due (via Angel) 10/2 Item Analyses Ch. 5 10/7 Validity I Ch. 9 10/9 Validity II Ch. 9 10/14 Reliability I Chs. 7 & 8 (skim both) 10/16 Reliability II Chs. 7 & 8 (skim both) 10/21 QUIZ 3 & Data Collection Day 10/23 SPSS Tutorial I Data Entry *Meet in MMC 115* 10/28 SPSS Tutorial II Data Analysis *Meet in MMC 115* 10/30 Ethics and Professional Issues I Ch /4 Ethics and Professional Issues II Ch /6 QUIZ 4 11/11 VETERANS DAY NO CLASS Rough Draft Methods Due (via Angel) Bring in Copies of Final Scales 11/13 Scale Revision Day Bring in SPSS Output 11/18 Review of Commonly Used Tests: The GRE Ch. 12 Rough Draft Results Due (via Angel) 11/20 Review of Commonly Used Tests: The Strong Interest Inventory Rough Draft Discussion Due (via Angel) 11/25-27 THANKSGIVING BREAK NO CLASS 12/2 QUIZ 5; Course Evaluations 12/4 WORK ON POSTERS Bring to class 12/9 POSTER PRESENTATIONS FINAL PAPER DUE (via Angel) 12/11 MAKE-UP EXAM