Distance Learning Master of Cost Estimation and Analysis Degree Program Assessment Plan

Transcription

1 Distance Learning Master of Cost Estimation and Analysis Degree Program Assessment Plan The Distance Learning Master of Cost Estimation and Analysis (MCEA) Degree Program is a joint degree program, combining academic efforts from the Naval Postgraduate School (NPS) and the Air Force Institute of Technology (AFIT), sponsored by the Naval Sea Systems Command (NAVSEA) and the Naval Air Systems Command (NAVAIR). Both schools have a vested interest in monitoring and evaluating students and curriculum to ensure highly qualified graduates are academically ready and able to meet the complicated cost estimation and analysis requirements expected by the sponsors. Ultimate responsibility for monitoring students and evaluating the program lies in the school that manages or hosts the annual cohort of students; NPS and AFIT will alternate hosting a cohort. When NPS hosts a cohort the MCEA program will incorporate a two tiered assessment system to monitor and evaluate student progress. This system tracks students throughout the program ensuring that course goals and program goals are met. The assessment program also ensures that students who are struggling with content are identified early and provided opportunities to remediate. The MCEA Program is both joint and interdisciplinary within NPS and AFIT, and consists of 16 courses. When NPS hosts a cohort the breakdown of course teaching responsibilities is as follows: Two Systems Engineering courses taught by the AFIT Systems Engineering Department Five Cost courses taught by the AFIT Cost Estimation Department Three Business courses taught by the NPS Graduate School of Business and Public Policy (GSBPP) Four Operations Research (OR) courses taught by the NPS OR Department Two Capstone courses facilitated by the NPS OR Department Course Assessment Activities Every course in the MCEA program has course goals. These goals communicate to the students that will be expected of them in the course. The goals also serve as the basis for the assessment of students. All courses employ a series of quizzes, exams, projects, presentations, or papers that require students to integrate the content they have learned for the weekly modules (in asynchronous courses) or weekly lessons (in synchronous courses). Both AFIT and NPS incorporate a final exam week to facilitate a comprehensive final exam or final project. Instructors are responsible for evaluating students on a frequent basis during the academic quarter to assess

2 progress. Students failing to progress at a minimum rate are reported to the MCEA Academic Associate for counseling and assistance. At the end of the quarter students are afforded the opportunity to provide feedback to the instructor on the entire course via the Student Opinion Form (SOF). SOF s are completed online and are mandatory; each SOF is submitted anonymously. Students must complete a SOF for each instructor or they will not receive their course grade. SOF s are composed of 16 statements that are completed using a fivepoint Likert scale. Students also have the option of adding free response comments concerning the course and the professor. The SOF results are provided only to the course professor and the Department Chair. The comments posed are as follows: 1. The course was well organized. 2. Time in class was spent. 3. The instructor seemed to know when students did not understand the material. 4. Difficult concepts were made understandable. 5. I had confidence in the instructor s knowledge. 6. I felt free to ask questions. 7. The instructor was prepared for class. 8. The instructor s objectives for the course have been made clear. 9. The instructor made the course a worthwhile learning experience 10. The instructor stimulated my interest in the subject area. 11. The instructor cared about student progress and did his/her share in helping others to learn. 12. Overall, I would rate this instructor: 13. Overall, I would rate this course: 14. Overall, I would rate the textbook(s): 15. Overall, I would rate the quality of the exams: 16. Overall, I would rate the laboratories: Two weeks after a quarter has ended, MCEA students are asked to complete an anonymous survey, which is facilitated using SurveyMonkey. The survey asks nine open ended questions that allow students to express their opinion of the courses and instructors that they just completed. The MCEA Program Managers and Academic Associates assess the survey results to make adjustments in future course offerings. The survey questions are as follows: 1. What quarter have you just completed? 2. What degree program are you enrolled in? Choose the best answer 3. What concepts did you find most valuable from last quarter's classes? What made them valuable? 4. Assess and describe the positive aspects of your instuctors. 5. Which, if any, of last quarter's classes did you find least useful and why? 6. What recommendations would you suggest to your instructors to improve the courses they taught?

3 7. Assess and describe the areas, if any, your instructors need improvement. 8. What single issue about last quarter would you most like to see changed and why? 9. Is there anything else you would like to say about your experience(s) so far in the MCEA program? During the third week after a quarter has ended the MCEA Academic Associate will assess each student s Graduate Quality Point Rating (GQPR, aka GPA). All students with a cumulative GQPR less than 3.0 will be considered academically deficient, and their names will be placed on the Academic Probation list. The MCEA Academic Associate will counsel those students on the Academic Probation list. To facilitate counseling students will be sent a self assessment form, which will be used during the counseling session. The self assessment form is found in NAVPGSCOL INSTRUCTION C, dated 17 July 2009 Internal Program Assessment The first time an internal assessment of the entire MCEA program can be accomplished is during the last two quarters of the curriculum when students are engaged in the two course Capstone Sequence. In these two courses the students apply everything they have learned in the previous six quarters. The course instructor will identify student teams, project topics, and government agency sponsor. There are two Project advisors, one a professor from NPS/AFIT and one from the sponsoring activity. Capstone Projects include three stages of grading per course. Teams must pass each stage prior to moving to the next stage. Stage 1: Project Assessment #1 Teams develop their plan of action, identifying the scope of the Estimate and the breakdown of the project cost elements. Early identification of data sources is emphasized with the help of the project sponsor. Assessments are provided in a page point paper. Papers are evaluated by the NPS/AFIT team advisor and project sponsor representative. A letter grade will be assigned to the Project Assessment #1. A team that fails must resubmit their project assessment. Stage 2: Project Assessment #2 Teams incorporate instructor feedback from the Project Assessment #1 and the assumptions and ground rules that they have determined for the estimate of the project. The requirement is a 6 10 page point paper. A letter grade will be assigned to Project Assessment #2. A team that fails must resubmit their project assessment.

4 Stage 3: Team Brief Teams brief the ground work for the foundation of their estimate in a power point presentation. This brief needs to reflect the feedback from Project Assessments #1 and #2, and also an assessment of the data collected. A letter grade will be assigned to the Team Brief. The course grade will be awarded based on the assessment of the three stages. Stage 4: Document the Estimate Documenting the estimate requires the student to describe their cost model, apply data to the model to generate an estimate, and then conduct sensitivity analysis to determine the models robustness. Written reports are submitted to the team advisors who provide feedback as soon as possible for the teams to incorporate in their Project Briefs.. A letter grade will be assigned to the written report. A team that fails must resubmit their written report. Stage 5: Project Brief Student teams will brief the entire assembly of students and advisors. The briefs will encompass the Estimate in Stage 4. Briefs will be scheduled for 40 minutes with 15 minutes available for questions, discussion, and feedback. A letter grade will be assigned to the Project Brief. Stage 6: Final Project Written Report The final project written report is the culmination of the previous five stages. As a minimum it should describe the final cost estimate that will be forwarded to the project sponsor. It includes the background of the project, the assumptions used to develop the cost model and to analyze the estimate, a discussion of the data source, and a recommended cost estimate. A letter grade will be assigned to the written report. The final grade will be awarded based on the Advisor s assessment of Stages 4 through 6. External Program Assessment Program evaluation and assessment are the keys to the success of all Academic Programs within the Naval Postgraduate School (NPS). Program evaluation and assessment requires dialogue between the sponsor and the MCEA Program Managers. This dialogue evolves based on the needs of the sponsors and stakeholders. These sponsors are kept informed via frequent informal direct contact with the MCEA Program Managers. In addition, twice a year there is a formal review called an Internal Program Review (IPR) where the sponsor assesses

5 progress in the curriculum development and execution. The agenda is ad hoc and depends on the cumulative effects of the informal direct contacts. NPS mandates that its curriculums undergo a comprehensive curriculum review once every two years. The main goal of the curriculum review is to assess the curriculum in regards to meeting the sponsor approved Educational Skills Requirements. Guidance is provided in the school policy instruction described in NAVPGSCOL INSTRUCTION E, dated 21 May The results of the review are submitted to the Operations Navy, Manpower and Personnel (N1). Summary The assessment activities described above are the essential elements in maintaining a high quality, essential program to benefit both the Department of Defense and the taxpayers. In summary, students evaluate the faculty and the program; MCEA and faculty evaluate the students and the program; and Sponsors evaluate the program. The MCEA faculty are dedicated to ensuring the MCEA program remains relevant and challenging. It will stay relevant because of the professionalism of all concerned. It will remain challenging because the work that needs to be performed by those who graduate from the program will be demanding. And, it is the responsibility of NPS and AFIT to ensure its graduates are fully prepared to do what is asked of them in service to the men and women in uniform who stand ready to defend our nation.