Student Assessment - A Body of Research and Practice

Transcription

1 CRLL International Symposium in Lifelong Learning Glasgow Caledonian University, Scotland, UK Overview of US Educational System Past 40 years of research Present research at SUNY Empire State College Future research your ideas and work Nan L. Travers, Ph.D. Director Collegewide Academic Review Organized 40 years ago for adult learners Open Curriculum - All undergraduate and some graduate students self-design their degrees Prior learning assessment is part of the degree design process Over 20,000 students 50% on-line and around the world (in all 50 states and over 40 countries) Governance and funding of Higher Education Governance controlled locally within state Funding combination tuition, endowments, state, and federal Accountability through Regional Accreditation Agencies Curricula control Faculty controlled at local level CAEL Standards Regional Accreditation Agencies Some states have creating systems for supporting and/or evaluating PLA (e.g. Oklahoma, Minnesota, Pennsylvania and Vermont ) Most states - the local institution governs PLA College Entrance Examination Board (College Board) 1930 s created College Level Examinations Program (CLEP) for entrance into universities Later - Advanced Placement Program American Council of Education (ACE) credit for military experiences (both secondary (General Education Exam GED) & postsecondary exams) Evolved to evaluating military and other professional experiences for credit The Educational Testing Service (ETS) 1947 began developing series of examinations 1

2 1968 and 1974, thirteen colleges of higher education were founded (or expanded) Adult learners as the primary focus Prior learning assessment Examples: Charter Oak College of CT College of Lifelong Learning (now Granite State College, NH) Community College of Vermont Minnesota Metropolitan State SUNY/Empire State College, NY Thomas A. Edison State College of New Jersey Commission on Non-Traditional Study (1971) Carnegie Corporation, Educational Foundation of America, Educational Testing Service (ETS) and College Board Examined current practices for non-traditional education and recommended new approaches The Cooperative Assessment of Experiential Learning (CAEL) Project ( ) Administered by ETS, funded by Carnegie Corporation; Organized by Morris Keeton CAEL Project ( ) Interviewed students, faculty, administration at 50 institutions on adult learner practices and prior learning assessment Pilot projects and assessor training initiatives were launched via mini-grants to various institutions. Results were written up as a series of working papers and piloted in more than 80 institutional projects. In addition, CAEL organised meetings twice a year to disseminate research findings and gather further feedback 10 CAEL Standards are born National Institute on the Assessment of Adult Learning: Thomas Edison State College of NJ National Center on Adult Learning (NCAL): SUNY Empire State College (ended in late 1990 s). Institute for Research and Assessment in Higher Education (IRAHE) at the University of Maryland University College Limited research studies sources Dissertations - 36 CAEL - 10 Institutional research 7 Reviewed Journals - 6 Three major themes PLA Program Practices Institutional Outcomes (Persistence, Graduation Rates, GPA, etc.) Student Outcomes PLA Policies State Regional Accreditation Institution Program Structures Barriers Enablers Best Practices Assessment Practices Credit application Assessment Processes Faculty Development Program Review 2

3 Institutional mission and commitment Institutional support (administrative, faculty, financial) Program factors (student expectations, ways in which learning is evaluated, ways in which credits can be used, etc.) Assessor training Program feedback and evaluation Travers & Evans (2011). Evaluating prior learning assessment programs: A suggested framework, International Review of Research in Open and Distance Learning, Vol. 12, No. 1. Advising and assessing models: Pre-determined standards Open-standards Uncertain about the meaning of CAEL standards Had little knowledge about assessment Relied on personal knowledge, theoretical understanding and own experiences Lack of training most important areas identified: Understanding expectations (student and assessor) College-level learning Institutional Measures (single institutional studies) Retention, Persistence, Graduation Rates, GPA Career Opportunities, Salary Increases CAEL study (2010) Examined student data from 48 institutions, across US and Canada Significant increases based on PLA participation students had: Higher graduations rates, Persisted longer Took more credits at institution than non-pla counter parts Participation in PLA increases: Reflection Problem-solving and tacit knowledge Self-regulation/self-awareness/selfdirection Study Skills Understanding of the role of faculty/mentors Clear alignment with mission, philosophy, policies of institution Faculty and evaluator training Assessment practices College-level learning Assessing Learning Program Evaluation Focused on the student, faculty mentor and assessor experience Understanding knowledge construction and its assessment Sustainable program practices at large scale 3

4 Audience: Student, Peer Reviewers, Institution Voice: Professional authority, observer/reporter, editor, student voice, outside authority Presentation of the learning: Learning distinguished from experience, learning within different context, learning within different fields Vocabulary: Cultural, Field Specific and Educational Glossing Theory: The application, demonstration and context within which the concepts are formed and function. Communication: The ability to engage in and use sources of communication to share a discourse of thoughts, opinions and information. Relationship Among Ideas: The ability to shape, engage and interpret ideas to provide analytical and critical perspectives, strategies, abstractions, and synthesis and to explore those ideas through focused questions and discourse. Utilization of Ideas: The ability to apply and demonstrate ideas and concepts within experiential contexts. Understand Self in Relation to Ideas and Learning: The ability to self-regulate and own one s learning through reflection, suspension of disbelief, intellectual honesty and goal setting and to use selfregulation to engage the learning process. Understand Self, Learning and Ideas in a Broader Context of the World: The ability to bring contextual awareness from an expanded viewpoint to the learning with the awareness of and appreciation for the perspectives of others. Agreement and Validity within the Field and Higher Education: The learning is related to historical and field relevant perspectives and is consistently viewed as college-level across experts within a field and/or multiple faculty members. The evaluator and student experience: Examining the assessment process (Evaluator review of materials, student/evaluator assessment interview) Impact of PLA participation on self-regulated learning Examining changes in self-regulated learning and self-perceptions pre-post PLA participation eportfolios Student development PLA as a learning process Assessments of knowledge construction. Open Learning Resources Evaluating student learning Providing PLA resources for students, mentors and assessors 4

5 Moved into the Relational Age (and out of the Information Age) People can easily pick up information and connect with others Knowledge formation is at our fingertips PLA will play a significant role Students will come with more and more existing knowledge Need greater understanding around knowledge organization/construction and its assessment Interrelationships among knowledge is critical to assess Research in PLA critical to leverage: Recognition & Support Faculty Buy-in/ Faculty Development Access for students Best practices Learning Counts: CAEL online PLA Center YOUR RESEARCH!!!!! Peer reviewed e-journal Call for submission will be out in June First edition released Fall 2011 Capture what the students know use this back within the institution Change in students through process Use of RPL to what extent and how Use on-line RPL tools Faculty perceptions of student performance with vs without RPL Funding and return of investments costs & benefits Richness of knowledge how can use it? Evaluation audience & voice How do we see assessment rich process for everyone involved learning process How does the learner fit within the research and practice? Research can inform practice and vs.versa Need to write up our practices as scholarship Action research informed practice capture changes Quantitative and qualitative both Have all the right tools how do we talk about how we use them? Philosophical theories of practice psychological/cognitive need to really look at these relationships in bigger perspective macro questions Nan L. Travers Director of Collegewide Academic Review Empire State College 1 Union Ave. Saratoga Springs, NY Nan.Travers@esc.edu 5