Quality Assurance in Online and Hybrid Courses

Transcription

1 Quality Assurance in Online and Hybrid Courses

2 Presenters CSU-Northridge Kim Kirner, Assistant Professor of Anthropology Lori Campbell, Assistant Professor of Sociology Bonnie Paller, Director of Academic Assessment and Program Review Li Wang, Instructional Designer Contributed by Charlene Hu, Director of Instructional Development, CSU-Bakersfield

3 Outline Quality Assurance Needs Course Redesign Institute Campus Collaboration Scholarly Teaching

4 Quality Assurance Needs National Trend Quality Assurance Needs Course Redesign Institute Campus Collaboration Scholarly Teaching Chart copied from Sloan-C report: Changing Course: Ten Years of Tracking Online Education in the United States

5 Quality Assurance Needs CSUN Context Online Student Head Count

6 Quality Assurance Needs Context Continued Percent Online Students

7 Quality Assurance Needs Fall 2012 Models of Quality Assurance F2F Course Evaluations Online and Hybrid Courses Online Programs (State) Tseng College (Outreach) Fall 2012: 2749/6300 Courses in Moodle (44%) 173 Courses Fully Online

8 Course Redesign Institute Quality Assurance Needs Course Redesign Institute Campus Collaboration Scholarly Teaching Pedagogy Technology Absorb, Reflect, Practice

9 Course Redesign Institute - CSUN Quality Assurance Needs Course Redesign Institute Campus Collaboration Scholarly Teaching Summer Institute Schedule o 2-week f2f Institute o 8-week online o Showcase Institute Curriculum o Rubric as the Backbone o Metaevaluation Tool and Roadmap o Templates for course redesign (Template 1. Template 2, Template 3)

10 Campus Collaboration Quality Assurance Needs Course Redesign Institute Campus Collaboration Scholarly Teaching Academic Assessment Faculty Development Faculty Technology Center Universal Design Center University Library Partnership in Quality Assurance Effort

11 H/O/T Assessment Plan Administrative Process Funding through Undergraduate Studies/Academic Affairs Demarcating course types Calling for Proposals Designing assessment plans Implementing assessment 11

12 H/O/T Readiness Survey 12

13 Faculty Design Phase Identify course SLO to be assessed Identify section types to be used (H/O/T) Identify faculty if not same as investigator Identify or create embedded signature assignment Begin to identify or create rubric 13

14 Samples of SLOs Students will demonstrate knowledge of typical and atypical physical development from birth through adolescence (Special Education) The ability to make investment decisions using the time value concept of money; risk and return concept; and financial calculators (Finance) Students will be able to identify, describe, and evaluate examples of the four functions of management (Management) 14

15 Implementation Phase Students will demonstrate knowledge of typical and atypical physical development from birth through adolescence (Special Education) Traditional Class N=22; avg. 19.8/25 pts. Online Synchronous Class N=18; avg. 21.7/25 pts. Online Asynchronous Class N=13; avg. 23.3/25 pts. 15

16 Scholarly Teaching Quality Assurance Needs Course Redesign Institute Campus Collaboration Scholarly Teaching Anthropology Sociology Hybrid courses Redesigned through the Institute

17 Anthropology 222: Visions of the Sacred Why Redesign? PROBLEM: o Different skill levels of students o Lack of class time for more critical assignments SOLUTION: o Flip the classroom! o Lectures and quizzes go online o Homework and activities get done face-to-face Kim Kirner, Ph.D. Department of Anthropology CSUN

18 Online Lectures Students watch short lectures arranged in module series at their own pace in EchoCenter or YouTube.

19 Online Quizzes Allow the use of new photos and videos for interpretation based on concepts presented in lecture. The following image is a burial of 9-month-old twins from 27,000 before present found in Austria. The skeletons were found under a mammoth bone. What in the image indicates the strongest potential evidence of religious belief? a. None of these is evidence of potential religious belief b. The infants were placed under a bone. c. The infants are covered in red ochre. d. The infants were buried together. e. The infants were buried rather than cremated. Alt-text is provided for visually impaired students.

20 Interfaith Project Journal In preparation for a final descriptive paper in an unfamiliar religious event, students keep an online research journal.

21 Backward Design Step 1: Integrating departmental program learning outcomes, course student learning outcomes, and course content. Departmental program learning outcomes. Each of our courses is mapped onto the PLOs in a big matrix, ensuring consistency from instructor to instructor. Course student learning outcomes are expected to closely align. Program Learning Outcome Course Learning Outcome Topic Numbers Assessment Tool Recognize characteristics of human diversity across space and time from an anthropological Explain how religious systems are situated in diverse places and times, 2-15 Quizzes and Exams perspective (1) providing specific examples Discuss the concept of culture as a Describe how religious systems 1-15 Quizzes and fundamental principle in anthropology (4) Identify the causes and consequences of cultural diversity, social inequalities and change in human societies (5) Discuss anthropological theories and paradigms, how they have changed over time and how they are applied to explain fundamental aspects of the human condition such as cultural diversity and social change (6) Demonstrate the ability to conceptualize, collect, describe, analyze and interpret anthropological evidence according to generally accepted professional practice (7) Discuss ethics as they pertain to 21st century anthropology (8) Explain how anthropology may be used to engage in contemporary issues (9) Develop effective communication using anthropological standards (10) relate to other facets of culture Identify the causes, consequences, and processes of religious diversity and change Describe how anthropological theories about religion have changed over time Practice collecting data about one s own and others religious ideas and practices using basic qualitative methods Describe the basic ethical issues of studying religion anthropologically Discuss current issues in religion using an anthropological perspective Use writing and speaking to describe one s observations and participatory experiences while studying religion Exams Quizzes and Exams; Interfaith; Portfolio 1-8, 10-12, 15 Quizzes and Exams; Portfolio 2-15 Interfaith; Portfolio 1 Portfolio Interfaith; Portfolio 2-15 Interfaith; Portfolio I then align the course student learning outcomes with the relevant topics, allowing assessment using specific modules. These are linked to assessment tools.

22 Backward Design Step 2: Crafting module-specific learning objectives, which ensures students understand what each lecture s goals are. Module/Week: 2 Topic: Myth and Cosmology Starts with bringing relevant course SLOs over as objectives. Builds contentand processrelated objectives to provide goals for a lecture or activity. Course Objectives 1: Give examples of religious systems in diverse places and times 2: Describe how religious systems relate to other facets of human life 3: Describe how anthropological theories about religion have changed over time 4: Practice collecting data about religion through interviewing, participant observation, and phenomenological methods 5: Use writing and speaking to describe one s observations and participatory experiences while studying religion Module Objectives Assignment/Due Date Interaction and Community (Activities) Define myth as opposed to other types of literature Describe types of myths held in common by many cultures Apply ways of interpreting myths to specific examples Describe the functions of myth in the human experience Define cosmology and describe how it relates to the human experience Provide diverse examples of myths from various cultures and religions Relate myth and cosmology in specific cultural examples Portfolio Activity #1: Interpreting The Story of Taliesin (due in class) Quiz (due on Moodle by following week class) Interfaith Project Activity #2: Myth/Ritual (due online via Turnitin by week 5 class) F2F Reading discussion groups Portfolio Activity Online Lecture Quiz Interfaith Project Activity Resources and Materials Course Reader Overview Chapter 2 (online) E-Reserve Article: Leonard and McClure (online) Performances of The Story of Taliesin (in class) Finally, crafts a lesson plan.

23 Making an Assessment Plan: Year 1 SLO#1 (basic content knowledge); SLO#5 (intermediate content knowledge); SLO #7 (basic process knowledge) Type of Assessment Description Exact Direct Assessment Measurement Modules Assessed SLOs Demonstrated Quiz Online; multiple choice and matching; 20 questions (timed, 2 attempts allowed) Interfaith Project Field Note Journals: Online; report online research that links module content to a religious or magical tradition of the student s choice; demonstrates both content and process knowledge Quiz Quiz Quiz (combined) Journal and 7 Journal and 7 Final Descriptive Paper: Online; demonstrates cumulative knowledge of the religious or magical tradition and process knowledge of participant observation and non-biased event description and analysis Final Project (combined score of final descriptive paper and all journal entries) cumulative 7 Portfolio Project Portfolio: Hard copy based on in-class activities; demonstrates engagement with process knowledge combined with content knowledge (i.e., anthropological techniques of data collection and analysis related to specific content); final summative paper allows student to showcase their three most meaningful activities and describe what they learned in the course Final Portfolio Project (combined score of all weeks in class with a summative paper) Indirect Assessment Students describe most meaningful activities and what they learned cumulative 7 2, 6, n/a

24 Direct Assessments Demonstrating SLO achievement; ensuring comparable outcomes to f2f classes; identifying problems SLO #7: Practice collecting data about religion using anthropological methods SLO #5: Identify the causes, consequences, and processes of religious diversity and change SLO #1: Give examples of religious systems in divers places and times Final Project (%) Passed Journal Entry (%) Average Quiz (%) Class Average for SLO (%)

25 Indirect Assessments (QOLT) Identifying problems related specifically to online teaching Scale (1-5) 2.4: Assignments helped me learn the topics. 2.3: How activities helped me achieve learning goals each week made sense. 1.2: The purpose and format of the course were clear to me. Scale (1-5) 1.1: How to get started in the course (basic materials) were clear to me

26 Increasing Student Effort & Engagement by Flipping the Classroom Sociology 364: Social Statistics Why Redesign? Introduce more rigor Cover more material & in greater depth Get students to put forth more effort o Avg. time spent studying has declined to <14 hrs/week 26 Lori Ann Campbell, Ph.D. Department of Sociology CSUN

27 Most lectures go online o Online lecture- interactive format (Softchalk) o More complex topics covered f2f Frequent, low-stakes assessment (online) Use f2f classroom time for: o Problem-solving & interpretation of results o Discussions of appropriate research methods o Reading statistical research in sociology Strategy: Flip the Classroom 27

28 Three Types of Assessment Used After online lecture, students assessed online o Learning Check: lower-order cognitive skills After f2f class, students assessed online o Homework: higher-order cognitive skills Every other week: students assessed f2f Quiz Frequent, Low-Stakes Assessment Holds Students Accountable 28

29 Online Assessment: Providing Feedback Quickly 29

30 Using assessment data to inform teaching & further assessment Based on the Learning Check: 27% of students didn t understand r2 30

31 Benefits & Drawbacks of Hybrids? For the instructor: Initial start-up: time-consuming No video lectures so far Organization & punctuality key Combating student cheating For the student: Time commitment Difficult for disorganized students Requires discipline 31

32 Thank You. Kim Kirner, Lori Campbell Bonnie Paller, Li Wang, Your Feedback is greatly appreciated.

33 Quality Assurance in CSUB Call for online course nomination 10 courses nominated 2 course certified 4 course going through certification Summer Institute for Online Teaching 43 faculty received Applying QM rubric training Interest in rubric and diligent with feedback Charlene Hu Director of Instructional Development CSU-Bakersfield ([email protected]) Faculty Professional Development 3 faculty received training to become peer reviewers