Program Planning and Assessment (PPA) for Academic Programs. Comprehensive Review, Annual Review & Action Plan. Spring 2014

Transcription

1 Program Planning and Assessment (PPA) for Academic Programs Comprehensive Review, Annual Review & Action Plan Spring 2014 The purpose of Program Planning and Assessment at Hartnell College is to obtain an honest and authentic view of a program and to assess its strengths, opportunities, needs, and connection to the mission and goals of the college. The process is based on the premise that each academic program reviews assessment data and uses these data to plan for improvement. The results of these annual cycles provide data for a periodic (every five years) comprehensive review that shows evidence of improvement and outlines long-range goals. The Program Planning and Assessment process will improve and increase the flow of information about student learning, student success and student behavior at Hartnell College. The result of the process will also improve institutional effectiveness. Program/ Discipline Date Completed (must be Date Submitted in final form by 3/31/14)* to Dean 3/31/14 3/28/14 Astronomy *Please note that you should work with your colleagues and dean to ensure that this report is completed, revised as needed, in its final form and submitted no later than the end of March. List of Contributors, including Title/Position Pimol Moth, Astronomy Instructor This PPA report is organized in 3 sections and 11 subsections as follows: I. Comprehensive Review a. Overall Program Effectiveness, b. Instructional Staffing, c. CTE Programs Labor Market & Achievement, and d. Program Goals. II. Annual Review a. Course Data & Trends, b. Teaching Modality, c. Curriculum, d. Outcomes, and e. Previously Scheduled Activities. III. Annual Action Plan a. New Activities and b. Resource Requests. INSTRUCTIONS For programs/disciplines scheduled for comprehensive review in spring 2014, please complete Sections I, II, and III. For programs/disciplines scheduled for annual review, please complete Sections II and III.

2 II. ANNUAL REVIEW This section must be completed for ALL academic programs, including those scheduled for a comprehensive review in spring A. COURSE DATA & TRENDS 1. Please evaluate the 3-year trend of enrollment and success of courses in your program/discipline. Identify the courses you are choosing to examine this current year in the list below. You do NOT need to evaluate trends for each course every year. Course Number Course Name Does the course have any DE (online or hybrid) sections? AST1 Introduction to Astronomy Lecture yes AST1L Introduction to Astronomy Lab no Please use the data that have been provided. Analyze trends that you observe with respect to the data for the identified courses and answer the following questions. ENROLLMENT 2. Review the enrollment data. Describe and analyze any patterns or anomalies that you notice. What do you make of these patterns or anomalies? What actions should be taken to ensure continuous improvement? Course Fall 2010 Spring 2011 Fall 2011 Spring 2012 Fall 2012 Spring 2013 AST AST 1L Total From Fall 2011 onwards, enrollment in Astronomy has increased (compared to Fall 2010 and Spring 2011) because of the addition of an extra evening section, a distance education section, as well as a section offered at king City. The number of students enrolled in the sections has remained consistent since then. During Fall 2012 there was a drop of enrollment in the laboratory sections because one laboratory section had to be canceled. SUCCESS 3. Review the success data. Describe and analyze any patterns or anomalies that you notice. What do you make of these patterns or anomalies? What actions should be taken to ensure continuous improvement? 2 P a g e

3 Course Fall 2010 Spring 2011 Fall 2011 Spring 2012 Fall 2012 Spring 2013 AST-1 82% 65% 68% 71% 70% 74% AST-1L 88% 85% 90% 85% 87% 86% Total 84% 72% 75% 75% 74% 77% In general, there is a higher student success rate in the AST-1L (upper 80%) than in AST-1 lecture (low 70%). This is because the majority of the student grade in the lecture is based on exams whereas as the majority of the student s grade in the labs is based upon the laboratory write-ups. Students are in general more successful with their laboratory writeups than they are in exams. There is a notable decrease in the success rate of students from Spring 2011-Fall This may be due to inconsistencies in grading and standards between the different faculty members who were teaching the courses. This rate jumped to more consistent percentages during Spring Spring DEGREES AND CERTIFICATES 4. Describe the demonstrated effectiveness of the program over the past several years with levels and trends of achievement data, such as degree and certificate completions/awards. Since Astronomy is a new program (established officially in Spring 2013), students have not yet completed a degree in this program. However, the full time instructor has met with and mentored several students who expressed interest in majoring in Astronomy. It is anticipated that one of these students will complete his degree Spring 2014 with a declared major in Astronomy. He will be the first to earn an A.S. in Astronomy from Hartnell College. 3 P a g e

4 B. TEACHING MODALITY 1. Enter the number of Distance Education Courses, both fully online and hybrid sections, along with the number of full-time and adjunct faculty. Term No. of DE/ Online Sections No of Hybrid Sections Fulltime Faculty Fall Spring Adjunct Faculty 2. Compare student success in the DE teaching environment with success in the faceto-face teaching environment in the same course. Are there differences? To what do you ascribe the differences in your program? Discuss any other relevant factors regarding diverse teaching modalities and environments, such as specific locations. Course Fall 2010 Spring 2011 Fall 2011 Spring 2012 Fall 2012 Spring 2013 AST-1 82% 65% 68% 71% 70% 74% AST-Online N/A N/A 68% 63% 57% 60% Astronomy was first offered as a DE modality during Spring The data shows that the success rate in the online class is about 12-14% less than the face to face modality (except for Fall 2011 where it was the same). The main difference is that in the face to face lectures, students do much of their learning by listening to the instructor, whereas in the DE modality, the learning is done mostly through reading on their own. In the DE modality students are expected to spend the same amount of time on the class as in the face 2 face sections. However, because students who typically take DE classes have many external obligations, they may not have as much time to devote to the class compared to face 2 face students. 3. Describe the process to change and improve student success in DE courses/sections in your program, and any other relevant factors regarding diverse teaching modalities and environments, such as specific locations. In an effort to increase the success rate of online students, the instructor will encourage online students to come to campus and meet with the instructor in person, if possible. This will help the student clarify concepts that are difficult to visualize in the lecture and to discuss study tips. Also, the distance education committee will be creating a video for online students that they can watch to learn how to be an effective online student. This orientation video will help students to understand what they need to do to be successful. Also, the instructor will include more activities in the modules for students to review material that is in the reading and require students to participate in discussions (all discussions are currently extra credit). 4 P a g e

5 C. CURRICULUM Complete the following tables pertaining to courses scheduled for review. Courses scheduled for review during AY as previously specified Faculty member(s) responsible for coordinating (a) Was the course reviewed and (b) taken through the curriculum process? none N/A N/A N/A Date of approval (or anticipated approval) by Curriculum Committee Courses scheduled for review during AY Faculty member(s) responsible for coordinating AST45 Pimol Moth Fall 2014 AST1L Pimol Moth Fall 2014 Target semester and year Fa 2014 or Sp P a g e

6 D. OUTCOMES Use your Program Outcome Maps to assist you in this subsection. As you plan your course assessments, keep the higher level program outcome in mind. While course level assessment serves the purpose of examining the teaching and learning for that particular course, it also provides the data that will be viewed collectively for assessment of the associated program level outcomes. PROGRAM LEVEL OUTCOMES 1. Please complete the following tables. List Program level outcome(s) scheduled for assessment as previously specified Explain and discuss the impact and history of scientific theories and their importance in the advancement of astronomy. What changes have occurred in the program/discipline as a result of dialogue? More group activities will be developed to help students understand the important theories that involve the formation of the solar system and Universe. Also, there will be more practice questions for students to review using assessment tools such as Top Hat Monocle. For the online section, more videos and discussion questions will be included. We will also reassess this outcome in Spring 2015 for all sections to determine if the interventions were successful in increasing student understanding. Was the Program Outcome Assessment Summary completed? Yes 6 P a g e List Program level outcome(s) scheduled for assessment in AY Explain and discuss basic astronomical phenomena including, but not limited to: o Gravitation o The seasons o The phases of the moon o Eclipses Explain and discuss the impact and history of scientific theories and their importance in the advancement of astronomy. Have your course level SLOs needed for this program level outcome been assessed or scheduled for assessment? Yes Yes

7 2. Describe how program level outcomes were specifically addressed by the program/discipline during the past year. For example, were data gathered at the course level? Was there review and analysis of the data? How did the discipline faculty engage in discussion? Were any interventions conducted? Are there any plans to make changes to certificate/degree programs or improvements in teaching and student learning? Data was gathered from student learning outcomes that related to this program learning outcome for all sections and from all instructors (including adjuncts) teaching the course. This also included data from the online class. The faculty in the discipline met to review and analyze the data. In general, they found that students had a difficult time with this outcome. Students were able to name the theory, but were unable to identify the major concepts of the theory. They were able to describe bits and pieces of the theory, but were unable to synthesize it as a whole. The average score for all classes on this assessment standard was 49% in the face to face lecture sections and even lower (23%) in the online section. Instructors agreed that more group activities will be developed to help students understand the important theories that involve the formation of the solar system and Universe. Also, there will be more practice questions for students to review using assessment tools such as Top Hat Monocle. For the online section, more videos and discussion questions will be included. We will also reassess this outcome in Spring 2015 for all sections to determine if the interventions were successful in increasing student understanding. There are no plans to make changes in the degree program. CORE COMPETENCIES 3. Describe how Core Competencies were specifically addressed by the program/discipline during the past year. For example, were data gathered at the course level? Was there review and analysis of the data? How did the discipline faculty engage in discussion? Were any interventions conducted? Are there any plans to make changes to courses or improvements in teaching and student learning? Data was gathered from student learning outcomes that related to the core competencies for all sections and from all instructors (including adjuncts) teaching the course. Student learning outcomes were mapped to core competencies. It was found that the astronomy class student learning outcomes map mostly to two major core competencies: Critical Thinking/Problem Solving and Communication Skills (mainly in the form of written communication). The faculty in the discipline met to review and analyze the data and found that students need to improve their critical thinking and problem solving skills. The interventions that will be conducted are discussed in detail in the course level assessment reports. But to summarize, instructors will review some of these concepts more by using Top Hat Monocle, group activities, and demonstrations to helps students better visualize the concepts. 7 P a g e

8 COURSE LEVEL STUDENT LEARNING OUTCOMES 4. Please complete the following tables. List courses scheduled for SLO assessment as previously specified In what term was the course assessed? AST1 Spring 2014 Yes AST1L Spring 2014 Yes AST1 Online Spring 2014 Yes List courses scheduled for SLO assessment in AY AST1 AST1L Faculty member(s) responsible for coordinating Pimol Moth, Zoe Buck, Philip Deutschle Pimol Moth, Zoe Buck Was the Course Assessment Summary Report completed? Target semester and year Fa 2014 or Sp 2015 Spring 2015 Spring 2015 AST1 Online Pimol Moth Spring P a g e 5. Describe course level assessments results and how they will influence your plans moving forward. For the Astronomy 1 lecture course, it was learned that students have a more difficult time understanding the size scales of astronomical objects and understanding why different astronomical phenomena occur and relating important astronomical theories to how the solar system and universe works. That is students are good at remembering vocabulary and definitions, but have a hard time envisioning the big picture such as how the different astronomical objects are related to each other and connections between major concepts that collectively explain why certain phenomena happen. Because of these assessment results, instructors will now stress the size scales of objects by creating PowerPoint slides to show them size comparisons of various astronomical objects, bringing in everyday objects and comparing to size scales of the planets, and also showing a video on size scales of astronomical objects. Instructors will also model the size structure of objects in the Universe by taking a set of stacking boxes, painting them with astronomical objects such as solar system, galaxies, star clusters, and universe and showing how they relate to each other in size by fitting the smaller boxes into the larger boxes. Instructors will use group activities to enforce understanding of astronomical theories and discuss misconceptions about why certain phenomena occur and explain the true reasons behind them. For the Astronomy 1 laboratory course, it was learned that students were able to master their mathematical skills by the end of the course because they were able to use the correct equations rather successfully to calculate the answers to problems. They were also able to refine their writing and organizational skills as evidenced by the quality of the lab reports. However, the main weakness seen from the assessment was that students experienced difficulty interpreting information and trends from plots. Because of this assessment result, the instructors will now spend more time discussing components of the plots and have students practice drawing their own plots before starting on their lab work. It was also learned that the online student success rate was much lower than in the face to face sections for every student learning outcome. In the future, more attention will be given to

9 improving proper understanding of all material in the online class, which includes more video clips, more required assignments, a student online orientation video, and encouragement from the instructor for students to come on campus for office hours if possible. 6. Describe assessment activities that need to be strengthened or improved. What are the challenges to achieving these improvements? It was discovered during the dialogue between the different instructors that there were inconsistencies between how the questions were given to the students and also how the exams were given. In order to more clearly and fairly assess the student learning outcomes in the different classes a standardized process needs to be developed for all classes. For example, in the student learning outcome: Given images of astronomical objects, students will identify the object with its correct designation, in one section, students were able to meet the minimum expectation very well, but in the other sections, students almost met the minimum expectation but not quite. During the dialog, instructors realized that there needs to be a standard set of images that are to be used for the assessment. In the section that performed very well, there were fewer images used and the images were easier to identify. Also in the student learning outcome: Given equations of physics, students will apply them to obtain numerical results, it was learned that one instructor had given a practice exam or review of the final lab exam, whereas the other instructor did not. The students were more successful and almost met the achievement standard in the section that gave a review. This is because in both sections, they were allowed to use their notes and students in the section that were given the review were able to know what to expect on the actual exam. Also, there was one section that performed better than the other sections. In this section, students were allowed to use their book and notes on the Final exam. The major challenge is to have the different instructors agree to a single process of assessing these SLOs without impinging on academic freedom. 9 P a g e

10 E. PREVIOUSLY SCHEDULED ACTIVITIES This subsection focuses on activities that were previously scheduled. An activity can address many different aspects of your program/discipline, and ultimately is undertaken to improve or enhance your program/discipline, and keep it current. Activity scheduled 1. The offering of Computational Astrophysics Course as an AST45 Special Projects Course 2. The offering of Observational Astronomy Course As an AST45 Special Projects course or as an Experimental Astronomy course 10 P a g e What success has been achieved to date on this activity? This course will be offered as an internship to students during the summer. Students have applied and will be selected before April This was not able to be completed. What challenges existed or continue to exist? This course had to be changed to an internship because of the current trend to not offer Special Projects courses. The challenge is to be able to offer it as an AST45 Special Projects course since the current trend is not to offer these types of courses. Also, MIRA imaging software needs to be purchased before this course can be offered. Will activity continue into AY 14-15? No Yes, Summer 2015 Will activity continue into AY 15-16?* No Perhaps, if the course is successful in * For each activity that will continue into AY and that requires resources, submit a separate resource request in Section III. 1. Evaluate the success of each activity scheduled, including activities completed and those in progress. What measurable outcomes were achieved? Did the activities and subsequent dialogue lead to significant change in student learning or program success? Computational Astrophysics Course The full time instructor has been planning with a Hartnell alumni to offer the Computational Astrophysics AST45 Special Projects course over the summer. However, after further discussion, it was decided that it would be easier to offer it as part of Title V s Research Scholars Institute so that students can get paid for attending. Thus, everything is in place to offer this program to the students this summer. The Research Scholars Institute with a focus on Astrophysics program will occur from June 9 th -July 8 th at Hartnell College. The outcomes will be discussed after the program is completed. Observational Astronomy Course Since there has been discussion about either removing Special Projects courses altogether or revising them to change the nature of the course, this course was not offered. It will be postponed until Summer, 2015 when the AST 45 revision occurs during Fall Also, the department needs to decide whether to offer it as a Special Projects course or whether to offer it as a new experimental course in Astronomy. This decision will be made in Fall, 2014.

11 III. ANNUAL ACTION PLAN This section must be completed for ALL academic programs, whether scheduled for annual or comprehensive review in spring A. NEW ACTIVITIES This subsection addresses new activities for, and continuing new activities into, AY An activity can address many different aspects of your program/discipline, and ultimately is undertaken to improve, enhance, and or keep your program/discipline area current. A new activity may or may not require additional resources. Activities can include but are not limited to: NEW CURRICULUM FURTHER DEVELOPMENT OF THE PROGRAM OR SERVICE GRANT DEVELOPMENT AND PROPOSALS FACULTY AND STAFF TRAINING MARKETING/OUTREACH ENROLLMENT MANAGEMENT STUDENT SERVICES ADMINISTRATIVE SERVICES SUPPORT OPERATIONS FACILITIES 1. List information concerning new projects or activities planned. Please keep in mind that resources needed, if funded, would not be approved until spring 2015 and provided until FY Ongoing activities involving resources that will no longer be available from grant funds starting FY must be planned for appropriately. Activity 1. The offering of Observational Astronomy Course as a Special Projects course or as an Experimental Astronomy course Strategic Plan Goal(s) No. & Letter (e.g., 5A)* Priority 2A Related Courses, SLOs, PLOs, or goals Related Courses: AST 1 and AST 1L Related Program Outcome: Demonstrate proficiency in applying scientific procedures for making observations and measurements typical of modern astronomical research. Desired Outcome(s) This course will help students majoring in Astronomy to master observational and telescope skills and learn important astrophysical concepts.. Resources Needed Grant funding needed for the mentor if offered as AST45, but if it is an Experime ntal course, no grant funding will be needed. Person Responsib le Pimol Moth and mentor (possibly P. Deutschle) Estimat ed Date of Complet ion (can be more than one year in length) Summer 2015 Comments This activity was scheduled for Summer 2014 but will be postponed until Summer P a g e

12 *** Please complete this page for each new activity. *** 2. This item is used to describe how the new activity, or continuing new activity, will support the program/discipline. Consider: Faculty Other staffing Facilities Equipment (non-expendable, greater than $5,000), supplies (expendable, valued at less than $5,000), Software Hardware Outside services Training Travel Library materials Science laboratory materials a) Describe the new activity or follow-on activity that this resource will support. There are three new Astronomy declared majors. They will benefit by taking this course as part of their degree program and will receive training on telescope usage and image processing and reduction. b) Describe how this activity supports any of the following: 1) Core Competency 2) Program level Outcome 3) Course level Outcome 4) Program/Discipline Goal 5) Strategic Priority Goal This course will satisfy program outcomes #3 and #4 by allowing students to experience what it would be like to be a research astronomer. They will receive hands on training and practical experience typical of astronomy research by learning how to take their own observations and subsequently analyzing the images to understand the properties of astronomical objects. This course will address strategic priority 1, 2, 4, 5, and 6. Strategic Priority #1- Student success is addressed because after taking this course, students will be more prepared and successful when they transfer in STEM majors. Strategic Priority #2, #4, #5, #6- Student access, effective utilization of resources, innovation and relevance for educational programs, and partnerships with industry, business, agencies, and education are addressed because this course gives students access to the latest software and technology used in observational astronomy while partnering with Fremont Peak Observatory. c) Does this activity span multiple academic years? NO If yes, describe the action plan for completion of this activity. 12 P a g e

13 d) What measureable outcomes are expected from this activity? List indicators of success. Measurable outcome and indicator will be based on how many students who take the course will continue on to declare themselves as Astronomy majors. That is it will be measured by student retention and persistence in the majors. The success of this requested resource will be assessed two years from now when these declared majors are set to graduate (determined by whether they have remained in the program or not) and the number of new declared majors in Astronomy that have been inspired by this course. e) What are the barriers to achieving success in this activity? There are three barriers to achieving success in this activity: One is that this activity relies on being able to acquire Mira Pro, an astronomical data reduction software. If we do not have the resources to purchase this software for the instructor teaching the course and for the students to use, we cannot offer the course. Second is that we need a committed faculty member to teach the course. If there is no faculty member ready to teach the course, it cannot be offered. The idea would be to have P. Deutschle who currently teaches part-time at Hartnell College be the instructor for the course. Third is that in order to offer the course there needs to be compensation to the faculty member teaching the course. If it is offered as AST45, then grant funds must be used to pay the faculty member. The alternative is to offer it as an experimental course in astronomy. In this case, the funds would come from the district. If there is no funding to pay the faculty member, then we cannot offer the course. 13 P a g e

14 B. RESOURCE REQUESTS If new/additional resources are needed for your program/discipline, it is important that you identify them and project their cost, and that these resources and costs be considered through the College s integrated planning (governance, budget development, funding decision making, and resource allocation) processes. A resource is likely to be something needed to support an activity that you have identified in IIIA. above, in which case you must link the resource with a specific activity number (first column below). ). All resource requests completed in the various columns of a specific row must be linked to the new or continuing activity numbered on the first column of that same row. A resource could also be something necessary for your program/discipline to function properly to improve student learning, such as updated equipment in a classroom; in such case be sure to note that the resource is NOT tied to a specific activity. Activity No. 1. Special Projects Course in Observational Astrophysics Personnel Classified Staff/ Faculty (C/F/M)* Faculty member to teach the course (F) Suppli es/ Equip ment (S/E)* * Technology Hardware/ Software (H/S)*** Mira Pro Ultimate Edition Software (S) Contr act Servic es Training Travel Scie nce Lab s Projected Costs $ (software)(s) $ (faculty salary) (F) 2. Not tied to any activity Astronomy/Geology Lab Technician (C) $35,000/year(C) * Personnel: Include a C, F, or M after the amount to indicate Classified Staff, Faculty, or Manager. ** S for Supplies, E for Equipment. If additional supplies, for example, are needed for ongoing activities, this should be requested through the budget rollover process. *** H for Hardware, S for Software. 14 P a g e

15 APPENDIX A. Strategic Priorities & Goals (from Hartnell College Strategic Plan ) Priority 1: Student Access Goal 1A: Hartnell College will provide higher education, workforce development, and lifelong learning opportunities with seamless pathways to all of the college s present and prospective constituent individuals and groups. Priority 2: Student Success Goal 2A: Hartnell College will provide a supportive, innovative, and collaborative learning environment to help students pursue and achieve educational success. Goal 2B: Hartnell College will provide a supportive, innovative, and collaborative learning environment that addresses and meets the diverse learning needs of students. Priority 3: Employee Diversity and Development Goal 3A: Hartnell College is committed to 1) increasing diversity among its employees; 2) providing an environment that is safe for and inviting to diverse persons, groups, and communities; and 3) becoming a model institution of higher education whose respect for diversity is easily seen and is fully integrated throughout its policies, practices, facilities, signage, curricula, and other reflections of life at the college. Goal 3B: To attract and retain highly qualified employees, Hartnell College is committed to providing and supporting relevant, substantial professional development opportunities. Priority 4: Effective Utilization of Resources Goal 4A: To support its mission, Hartnell College is committed to the effective utilization of its human resources. Goal 4B: Hartnell College is committed to having its physical plant, furnishings, and grounds maintained and replaced in a planned and scheduled way to support learning, safety, security, and access. Goal 4C: Hartnell College will maintain a current, user-friendly technological infrastructure that serves the needs of students and employees. Goal 4D: Hartnell College is committed to maximizing the use and value of capital assets, managing financial resources, minimizing costs, and engaging in fiscally sound planning for future maintenance, space, and technology needs. 15 P a g e

16 Priority 5: Innovation and Relevance for Programs and Services Goal 5A: Hartnell College will provide programs and services that are relevant to the realworld needs of its diverse student population, while also developing and employing a culture of innovation that will lead to improved institutional effectiveness and student learning. Priority 6: Partnership with Industry, Business Agencies and Education Goal 6A: Hartnell College is committed to strengthening and furthering its current partnerships, in order to secure lasting, mutually beneficial relationships between the college and the community that the college serves. 16 P a g e