Non-Academic (Departmental) Assessment

Transcription

1 Non-Academic (Departmental) Assessment What is Assessment? Assessment at Virginia Western is an evaluation process to determine that we are meeting our mission to provide affordable, accessible, and quality educational opportunities and workforce training to meet individual, community and global needs. The assessment process provides faculty, staff and administration information about the quality of student work and the effectiveness of academic and educational support programs. Why do we need non-academic assessment? As an institution, our goal is to educate our students. As employees of the college, we have expectations that our departments are meeting the needs of our students, that the students are satisfied with the services we are providing and that our departments have a positive impact on the student learning environment. At Virginia Western, departments are assessed to determine if the services provided are meeting student needs and that the college is meeting the strategic goals of the Virginia Community College System (VCCS) and the college. Assessment of administrative, academic and student support services is a requirement of the Southern Association of Schools and Colleges (SACS), Virginia Western's accrediting agency. Therefore, as an institution, the college must assess its non-academic departments. Non-Academic Assessment at Virginia Western Departmental objectives have been developed for each department at Virginia Western and are assessed annually. These outcomes are described in the Institutional Effectiveness Plan (IE Plan) for the department. The IE Plan details the purpose of the department, how the department supports the mission through related institutional goals, the objectives for the department, and the methods used to assess achievement of the objectives. The supervisor for each department is in charge of assessment. Each year the supervisors lead the assessment efforts for their department by updating the IE Plan, coordinating the assessments, collecting and summarizing the results and documenting improvements based on the results. Assessing Departmental Objectives Step 1: Create/Update IE Plans When a new department is created, the supervisor for that area completes an IE Plan using the Non- Academic IE Plan Template found in this section. Once the department is established, IE Plans are updated annually by the supervisors and submitted to their respective VP or dean in mid-july.

2 The VPs and deans review the IE Plans for each department and meet with the supervisors to make changes if necessary. The VP or dean will save an electronic copy of the Non-Academic IE Plan to the appropriate Division Assessment Folder in the Institutional Effectiveness folder on the L Drive by the end of July. When IE Plans are created/updated, they should be simple and straightforward. Supervisors should avoid lengthy wordings and flowing descriptions. An outline of each piece of the IE Plan is listed below. Purpose Statements Each non-academic department must have a purpose statement. For non-academic departments, the purpose statement should include the parties served and the primary functions and core activities of the department. The purpose statement on the IE Plan should match the purpose statement on the college website or in the college catalog. Relation to Institutional and Divisional Goals The purpose of the department should be in support of one or more of the divisional goals which, in turn, support one or more of the institutional goals. Objectives Each non-academic department will need at least three departmental objectives. The objectives should include the results that are expected from the services provided by the department, measures that indicate if the department is effective, and other performance indicators that demonstrate how well the department is achieving its mission or purpose. Outcomes should directly relate back to the Purpose Statement for the department. Specific projects planned in support of an outcome may be listed in the Implementation section of the IE Plan. Guidelines for Departmental Objectives 1. Is the wording of the outcome easily understood? 2. Is the outcome measurable? You need to be able to establish clear guidelines for whether your program has achieved this outcome. 3. Is the outcome too detailed? If your outcome includes a list or uses conjunctions (e.g., and, or ), you will need to separately assess whether your area has met each part of the outcome. 4. Is the outcome relevant to the success of your area? 5. Can results from the outcome s assessment be used in your area to make improvements? Assessments Each departmental objective must be assessed on at least an annual basis. On the IE Plan under Assessments list each assessment you plan to use for the current academic year and indicate which objective will be evaluated using the assessment. 1. Each objective must have at least one assessment method.

3 2. Multiple assessment methods for each objective are preferred. Having only one assessment method is similar to having a course grade based on a single exam. 3. Direct measures are preferable to indirect measures. Where possible, use at least one direct measure as your primary assessment, and use indirect measures (such as most surveys) to supplement these results. When choosing assessment methods, you may want to consider some of the following selection criteria: Is the data relevant? Can it be used to show that the department is meeting its purpose and goals? Is the data useful? Is trustworthy data produced? Is the data valid and reliable? What is the development/preparation time involved in the assessment? What are the costs vs. benefits? It comes down to this: Can I use the data to make the decision? Is the data understandable? Does it map to the outcomes? It is recommended that direct assessment or a combination of direct and indirect assessments are utilized. Good Assessment Techniques should include assessments that are: Valid directly reflect the objective being assessed Reliable especially inter-rater reliability when subjective judgments are made Actionable results help staff identify what departments are doing well and what requires more attention Efficient and cost-effective in time and money Engaging to students and other respondents so they ll demonstrate the extent of their learning Interesting to staff and other stakeholders Triangulation multiple lines of evidence for each objective that point to the same conclusion Standards for Success Standards for success are the grading scale for whether your area has achieved success on the departmental objective. Be realistic the standards should be a reflection of your area s excellence, but still relatively achievable. These standards may need to be adjusted up or down in later IE Plans based on your area s history of results. The standard for success should include the level of performance level expected in terms of a metric and a percentage if appropriate. For example, 80% of financial aid notices will be mailed to students 30 days or more before the beginning of classes or $100,000 in donations will be collected. Implementation The implementation area can include any specific steps planned in your area to help achieve your outcomes. Specific projects are tools your area is using to achieve an outcome. For example, the Bookstore might choose to implement a new online system for student purchases. Having an online

4 system is not a purpose of the Bookstore it is a tool that the Bookstore is using to, say, help with college revenue and with student satisfaction. The objective should relate to these purposes (such as Increase college revenues by streamlining bookstore processes ); the project itself may be listed in the implementation section. Dissemination/Discussion Details of how assessments will be completed should be listed in this area. If you are using a test, survey, or other measurement tool, please reference it here and attach a copy of the tool to the IE plan. Details of when the results will be compiled and distributed to staff within the department and to the campus community should also be explained in this area. Step 2: Conduct Assessments Throughout the academic year, the assessments listed on the IE Plans are conducted. The data collected through the assessments are used to complete the Non-Academic Assessment Reports described below. Step 3: Analyze the Assessment Results Once the assessment has been conducted, the supervisor works with appropriate staff members to complete the Non-Academic Assessment Report. The template for this report is located in this section (separate Excel spreadsheet). The Non-Academic Assessment Report should be completed each June. Once the supervisor completes the report, it is submitted to the appropriate VP or dean for review. The VP/dean saves an electronic copy of the Non- Academic Assessment Report and any supporting documentation to the appropriate Department Assessment Folder in the Institutional Effectiveness folder on the L Drive by the end of June. The Non-Academic Assessment Report is an Excel file. To provide additional guidance, comments have been added to the form. Comments are added to every cell with a red dot in the upper right-hand corner. When you move your cursor to these cells, a message box will appear with additional instructions. There are three main parts to the Non-Academic Assessment Report: Background Information, Results and Changes Influenced by Assessment Results. Part 1: Background Information In this section, supervisors should provide basic information about the assessment report including the author, completion date and department. Part 2: Results

5 In this section, detailed information regarding whether or not the standard of success was achieved for the assessment should be provided. It is important to indicate the strengths and weaknesses of the department with regards to the assessment, based on the assessment results. Part 3: Changes Influenced by Assessment Results In this section, the improvements that will be made to the program in response to the results of the assessment should be listed. The timeline for implementing the changes should also be included. For example, if it is a change related to the graduation application process, it may be implemented in the following fall; however, a change to the commencement ceremony may not be implemented until spring. Step 4: Implement Changes and Document Improvements Once the Non-Academic Assessment Reports for a department are completed for the year, the information on the forms should be used to make changes to the IE Plan for the upcoming academic year by mid-july. Throughout the summer, any changes should be finalized and communicated to all staff in that department as well as the campus community, if appropriate. Evidence that improvements were made and were implemented should be submitted directly from the supervisors to the IEO as they are completed. This documentation should be sent electronically to ie@virginiawestern.edu. Documentation of improvements may include meeting minutes, conversations or forms that have changed from one year to the next. In other instances, a planned improvement may not be implemented for various reasons, such as a lack of funding or space, or a change in staff. If this is the case, an from the supervisor should be sent to ie@virginiawestern.edu explaining why the improvement was not implemented and describing future plans for the department. The IEO compiles the documentation for implementation of improvements collected each year.