Formative Assessment for Evaluation: Collaborative, community-based design of evaluation tools and systems

Transcription

1 Formative Assessment for Evaluation: Collaborative, community-based design of evaluation tools and systems Dr Amy Gullickson Hadeel Al-Nawab Centre for Program Evaluation September 2015 AES International Evaluation Conference 2015

2 Agenda Context Plan Components (Webinars and workshop)

3 Advanced Technological Education

4 Funding Levels Funding Levels Centers 177 Projects

5 Funding Levels 38 Centers 177 Projects

6 Streams of Activity Program Development and Improvement Curriculum Development Professional Development Targeted Research Leadership Capacity Building Teacher Preparation Business and Entrepreneurial Skills Development for Students

7 Discipline Areas

8 Roles Roles Principal Investigator (PI) Co-Principal Investigator (Co-PI) Staff Evaluator

9 Levels of Experience Principal Investigator (PI) Co-Principal Investigator (Co-PI) Staff Evaluator

10 Streams of Activity Program Development and Improvement Curriculum Development Professional Development Targeted Research Leadership Capacity Building Teacher Preparation Business and Entrepreneurial Skills Development for Students

11 Streams of Activity Streams of Activity Program Development and Improvement Curriculum Development Professional Development Targeted Research Leadership Capacity Building Teacher Preparation Business and Entrepreneurial Skills Development for Students

12 Discipline areas Discipline Areas

13 Discipline areas Discipline Areas Program Development and Improvement Curriculum Development Professional Development Targeted Research Leadership Capacity Building Teacher Preparation Business and Entrepreneurial Skills Development for Students

16 Discipline areas Program Development and Improvement Curriculum Development Professional Development Targeted Research Leadership Capacity Building Teacher Preparation Business and Entrepreneurial Skills Development for Students

17 Demo

18 FORMATIVE ASSESSMENT SYSTEMS FOR ATE COMMUNITY BASED DESIGN OF PROJECT EVALUATION TOOLS / SYSTEMS

19 VISION VISION A system of tools that ATE PIs can adapt and use to identify and collect real-time data for day to day learning and decision making on their projects. COMMUNITY BASED DESIGN OF PROJECT EVALUATION TOOLS / SYSTEMS FORMATIVE ASSESSMENT FOR ATE

20 Strategy Strategy

21 Streams of Activity Streams of Activity Program Development and Improvement Curriculum Development Professional Development Targeted Research Leadership Capacity Building Teacher Preparation Business and Entrepreneurial Skills Development for Students

22 Recipe for Success Recipe for Success Benefits for Participants Activity Benefits for the Project Logic Models (LM) Webinar 1 Workshop Recruiting Backwards Design Webinar 2 LM for Workshop Workshop

23 Recipe for Success Recipe for Success MOA for participants

24 Project Overview Webinars Workshop Synthesis Individual Project Across Projects Blogs Logic Model High Level Logic Models Strategy and Prioritising Evaluative Activities High Level Evaluation Data/Tool Needs System Tools, Activities, and Priorities

25 Leadership Team, Advisory Committee & Keynote Speakers Executive Group Arlen Gullickson Amy Gullickson Lori Wingate Gordon Snyder Dee Al Nawab Advisory Committee Elaine Craft Mike Lesiecki Joyce Malyn Smith Helen Sullivan ATE PIs & Senior Staff Workshop Keynote Speakers Janet Clinton John Hattie

27 Webinars Overview 85 56

28 Webinars Planning & Implementation Content Design Logistics Blackboard Online vs PDF versions

29 Blackboard Ba Chat Box

30 Recipe for Success Recipe for Success Benefits for Participants Activity Benefits for the Project Logic Models (LM) Webinar 1 Workshop Recruiting Backwards Design Webinar 2 LM for Workshop Workshop

31 Recipe for Success Benefits for Participants Activity Benefits for the Project Logic Model (LM) Webinar 1 Workshop Recruiting Backwards Design Webinar 2 Workshop Logic models and early identification of formative needs

33 WORKSHOP Workshop Moving toward Real Time Data for ATE Projects

34 Workshop Attendees Workshop Attendees Principal Investigator (PI) 11 Co-Principal Investigator (Co-PI) 6 Staff 4 Facilitator 5 Evaluator 10

35 Working Groups Working Groups

36 Working Groups Working Groups Program Development and Improvement Curriculum Development Professional Development Targeted Research Leadership Capacity Building Teacher Preparation Business and Entrepreneurial Skills Development for Students

37 Activity Structure Activity Structure Task 1 Logic Models Refining Individual Logic Models, Creating High Level Logic Models INPUTS ACTIVITIES OUTPUTS Facilitator Techniques Participant Expertise Each person/team gets a big post it and a pack of small post its. Use the big post it to create a logic model of your project or center in the activity stream you are assigned Logic Model for each participant / team Post -its 15 Minutes

38 Recipe for Success Benefits for Participants Activity Benefits for the Project Draft logic model Webinar 1 Workshop Recruiting Backwards design: identify data collection needs Refined logic model & data mapped onto management needs and project success throughout Webinar 2 Workshop Logic models and early identification of formative needs

39 Recipe for Success Benefits for Participants Activity Benefits for the Project Draft logic model Webinar 1 Workshop Recruiting Backwards design: identify data collection needs Refined logic model Data mapped onto management needs and project success throughout Webinar 2 Workshop Logic models and early identification of formative needs Logic models for activity streams, Prioritised needs for tools and systems, Collaborators & next steps

40 Project Overview Webinars Workshop Synthesis Individual Project Across Projects Blogs Logic Model High Level Logic Models Survey Evaluative Activities High Level Evaluation Data/Tool Needs Strategy, System Tools, Activities, and Priorities

41 Blogs

42 SURVEY Survey

43 Survey Results - Highlights SURVEY Results Highlights Value of Ideas Presented Outcome of Participation Subsequent Action

44 Next Steps Continued collaboration Dashboard development Common items for surveys Evaluation checklist for PD projects

45 Q&A

46 Thank you Copyright The University of Melbourne 2015